Augmented Reality Dental Design Method and System

ABSTRACT

A method and system for designing a dental appliance for an individual. A 3D model of the individual&#39;s features including a portion of their face and arches is displayed on a 3D display. The 3D model includes an augmented reality dental appliance. The 3D model can be manipulated by inputs detected by a motion sensor, a brain-computer interface, both, or other sensors. In response to gestures neural activity, or other inputs, the augmented reality dental appliance or other aspects of the 3D model are modified. The 3D model is updated in response to the modified dental appliance or other changes, and repositioned to provide an updated 3D model. The updated 3D model is displayed on the 3D display. This system and method facilitates modification of the augmented reality dental appliance and observation of the resulting aesthetic effects.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.15/120,348, filed on Aug. 19, 2016, which is a 371 of PCT ApplicationSerial No. PCT/CA2015/000101, filed Feb. 20, 2015, which claims thebenefit of priority of U.S. Provisional Patent Application No.61/942,734 filed Feb. 21, 2014, and U.S. Provisional Patent ApplicationNo. 62/075,665 filed Nov. 5, 2014, all of which are hereby incorporatedby reference in their entirety.

FIELD

The present disclosure relates generally to design of dental appliancesor restorations.

BACKGROUND

Currently, proposed dental appliances or restorations are visualized byeither trying in a replica of the restoration in the mouth of a subject,or by including 2D images of the subject in dental designer software.Examples include Densply's TruRx software for denture designs (see alsoUnited States Publication No. 2010/0076581) and 2D image arrangementfrom 3Shape for use on individuals with teeth (see also United StatesPublication No. 2013/0218530).

Densply's TruRx method is a commercially-available solution for digitalmodeling of a subject. This method involves placing reference indicia onthe face of the subject, positioning a mouth shield to cover at leastcover a portion of a subject's teeth thereby creating a voided area inthe following digital photograph of the subject's face. The softwareuses the reference indicia size in the photograph to compare dimensionsof the subject's face. The voided area is identified in the software andthe selected materials and structures for making the denture aresuperimposed on the voided area of the digital image so that apractitioner or the subject can see the results of what the subject maylook like with the selected combination.

SUMMARY

Herein provided is a system which integrates 3D imaging, dental designersoftware, and a 3D display to display hypothetical dental restorationsand options in real time, from any angle and perspective. The systemsenses inputs allowing a layperson individual to interact with acompound model of the individual's head and mouth and an augmentedreality (“AR”) dental appliance or restoration, by using inputsincluding one or more of movements and gestures, manipulation of asimple physical interface, or measurement of the individual's neuralactivity through a brain-computer interface (“BCI”) (e.g., anelectroencephalographic BCI, a magnetoencephalographic BCI, etc.). The3D display and the responsiveness to intuitive hand gestures or BCI dataof the 3D model facilitate use of this system by a layperson individual.The individual can select a variety of design options for the appliancesuch as tooth morphology, arrangement, and colour. The compound model isupdated in response to the design options, allowing the appearance of aproposed dental appliance to be confirmed and changed. The systemsimilarly provides a view of proposed dental restorations in a multitudeof facial expressions, lighting conditions, etc. BCI data may also becompared against empirical data of the individual in various emotionalor other involuntary states to assess the individual's preferences andprovide a suggested dental appliance.

The primary driver in design of a denture or other dental appliance isproviding a physiologically appropriate bite. Such a bite can beprovided to an individual by a variety of combinations of replacementteeth. While remaining at an appropriate position, the bite can becomposed of a variety of different combinations of shapes and sizes ofteeth (particularly where both upper and lower dentition are beingreplaced by dentures or other appliances). The particular choice ofdentition can have a significant impact on the aesthetic result (e.g. onthe resulting smile, etc.). It is, therefore, desirable to provide amethod and system which allow an individual to have meaningful inputinto the aesthetic presentation of a denture or other dental appliancebased on the size, shape, and/or orientation of the dentition includedon the appliance. It is an object of the present disclosure to obviateor mitigate at least one disadvantage of previous approaches todesigning dentures.

In a first aspect, the present disclosure provides a method and systemfor designing a dental appliance for an individual. A 3D model of theindividual's features including a portion of their face and arches isdisplayed on a 3D display. The 3D model includes an augmented realitydental appliance. The 3D model can be manipulated by inputs detected bya motion sensor, a brain-computer interface, both, or other sensors. Inresponse to gestures neural activity, or other inputs, the augmentedreality dental appliance or other aspects of the 3D model are modified.The 3D model is updated in response to the modified dental appliance orother changes, and repositioned to provide an updated 3D model. Theupdated 3D model is displayed on the 3D display. This system and methodfacilitates modification of the augmented reality dental appliance andobservation of the resulting aesthetic effects.

In a further aspect, the present disclosure provides a method ofdesigning a dental appliance for a subject individual includingdisplaying a 3D model of the subject individual on a 3D display, the 3Dmodel including a scanned feature comprising a dental arch of thesubject individual, and a portion of a face of the subject individualand the arch for relating the arch to the face; and an augmented realityfeature comprising a dental appliance for the subject individual;detecting an input with a sensor; modifying the dental appliance inresponse to the input to provide a modified dental appliance;repositioning the scanned feature in response to the modified dentalappliance to provide a repositioned scanned feature; updating the 3Dmodel in response to the modified dental appliance and the repositionedscanned feature to provide an updated 3D model; and displaying theupdated 3D model on the 3D display.

In an embodiment, the input includes a voluntary input.

In an embodiment, the voluntary input includes a gesture-based input.

In an embodiment, the gesture-based input includes gripping a feature ofthe 3D model on the 3D display and manipulating the feature.

In an embodiment, gripping the feature includes gripping the featurewith a hand.

In an embodiment, the feature includes dentition of the dentalappliance.

In an embodiment, manipulating the feature includes changing angulationof the dentition.

In an embodiment, the gesture-based input originates from the subjectindividual.

In an embodiment, the gesture-based input originates from a non-subjectindividual.

In an embodiment, the input includes a voluntary input.

In an embodiment, the voluntary input includes a gesture-based input.

In an embodiment, the sensor includes a motion sensor.

In an embodiment, the voluntary input includes a neural activity input,and the sensor includes a brain-computer interface.

In an embodiment, the neural activity input includes a conceptualizationof the modified dental appliance.

In an embodiment, the neural activity input includes a conceptualizationof modifying the dental appliance.

In an embodiment, conceptualization of modifying the dental applianceincludes conceptualizing gripping a feature of the 3D model on thedisplay with a hand and manipulating the feature.

In an embodiment, the feature includes dentition of the dentalappliance.

In an embodiment, manipulating the feature includes changing angulationof the dentition.

In an embodiment, the voluntary input includes a gesture-based input,and the sensor includes a motion sensor.

In an embodiment, the voluntary input includes a neural activity input,and the sensor includes a brain-computer interface.

In an embodiment, the neural activity input includes neural activityinput from the subject individual.

In an embodiment, the neural activity input includes neural activityinput from a non-subject individual.

In an embodiment, the input includes constraining at least a portion ofthe scanned feature to a target position, and the modified dentalappliance includes a modified feature which facilitates the targetposition.

In an embodiment, the target position includes a selectedmaxillomandibular relationship.

In an embodiment, the selected maxillomandibular relationship is a restposition, and the dentition provides a freeway space of between 1 and 4mm at the rest position.

In an embodiment, the selected maxillomandibular relationship is at aselected occlusal position, and the dentition provides occlusion at theselected maxillomandibular relationship.

In an embodiment, the modified feature includes dentition of the dentalappliance.

In an embodiment, the method includes detecting an involuntary inputwith the sensor; modifying the dental appliance in response to theinvoluntary input to provide the modified dental appliance;repositioning the scanned feature in response to the modified dentalappliance to provide the repositioned scanned feature; updating the 3Dmodel in response to the modified dental appliance and the repositionedscanned feature to provide the updated 3D model; and displaying theupdated 3D model on the 3D display.

In an embodiment, the involuntary input includes involuntary input fromthe subject individual.

In an embodiment, the involuntary input includes involuntary input froma non-subject individual.

In an embodiment, the involuntary input includes a neural activity inputand the sensor includes a brain-computer interface.

In an embodiment, the involuntary input includes a change in a facialexpression and the sensor includes an optical sensor.

In an embodiment, the method includes detecting an involuntary inputwith the sensor; correlating the involuntary input with a preferencecriterion and with the modified dental appliance to determine apreference of the individual; modifying the modified dental appliance toprovide a suggested dental appliance correlated to the preference of theindividual; repositioning the scanned feature in response to thesuggested dental appliance to provide a suggested scanned feature;updating the 3D model in response to the suggested dental appliance andsuggested scanned feature to provide a suggested 3D model; anddisplaying the suggested 3D model on the 3D display. In an embodiment

In an embodiment, the preference criterion includes an emotional stateof an individual.

In an embodiment, the preference criterion includes a voluntary input ofan individual.

In an embodiment, the involuntary input includes involuntary input fromthe subject individual.

In an embodiment, the involuntary input includes involuntary input froma non-subject individual.

In an embodiment, the involuntary input includes a neural activity inputand the sensor includes a brain-computer interface.

In an embodiment, the involuntary input includes a change a facialexpression and the sensor includes an optical sensor.

In an embodiment, the involuntary input is in response to the updated 3Dmodel.

In an embodiment, the 3D model includes a saved position, the savedposition having a selected scanned feature of the face.

In an embodiment, the method includes repositioning the scanned featureto the saved position; updating the 3D model in response to the savedposition and repositioned the scanned feature to provide a savedposition 3D model; and displaying the saved position 3D model on the 3Ddisplay.

In an embodiment, the scanned feature includes external feature data ofthe face for additional detail on the face in the 3D model.

In an embodiment, the external feature data of the subject individual'sface includes data for including substantially the entire face of thesubject individual's face in the 3D model.

In an embodiment, the method includes acquiring data of the scannedfeature.

In an embodiment, acquiring data of the scanned feature includesoptically scanning the scanned feature.

In an embodiment, acquiring data of the scanned feature includesultrasonographically scanning the scanned feature.

In an embodiment, acquiring data of the scanned feature includesacquiring additional data of the scanned feature in response to theinput and updating the 3D model to include the additional data.

In an embodiment, acquiring additional data and updating the 3D model toinclude the additional data are each performed continuously andsubstantially in real-time.

In an embodiment, adoption of a facial expression by the individualresults in updating the 3D model to include the additional data, andwherein the additional data includes external feature data of theindividual adopting the facial expression.

In an embodiment, the input includes a neural activity input, and thesensor includes a brain-computer interface.

In an embodiment, acquiring data of the scanned features includesconfirming that the subject individual is at a maxillomandibularrelationship corresponding to a rest position for the individual andacquiring data of the face when the maxillomandibular relationship is atthe rest position.

In an embodiment, confirming that the subject individual is at amaxillomandibular relationship corresponding to the rest positionincludes measuring jaw muscle activity of the individual to confirm amaxillomandibular relationship having a minimum energy usage.

In an embodiment, measuring the jaw muscle activity includes applyingelectromyography to the individual.

In an embodiment, confirming that the subject individual is at amaxillomandibular relationship corresponding to the rest positionincludes exhausting jaw muscles of the individual.

In an embodiment, exhausting jaw muscles of the individual includesapplying transcutaneous electrical nerve stimulation to the jaw muscles.

In an embodiment, data for displaying the 3D model includes data of theface when the maxillomandibular relationship is at the rest position.

In a further aspect, the present disclosure provides a system fordesigning a dental appliance for a subject individual including acomputer readable medium for storing a 3D model, the 3D model includinga scanned feature including a dental arch of the subject individual anda portion of a face of the subject individual and the arch for relatingthe arch to the face, and an augmented reality feature including adental appliance for the subject individual; a 3D display for displayingthe 3D model; a sensor for detecting an input; a processor operativelyconnected with the computer readable medium for processing the 3D model,with the sensor for receiving the input, and with the 3D display fordisplaying the 3D model, the processor configured and adapted to: modifythe dental appliance in response to the input to provide a modifieddental appliance; reposition the scanned feature in response to themodified dental appliance to provide a repositioned scanned feature;update the 3D model in response to the modified dental appliance and therepositioned scanned feature to provide an updated 3D model; and displaythe updated 3D model on the 3D display

In an embodiment, the sensor includes a motion sensor for detecting agesture-based input on the 3D model.

In an embodiment, the sensor includes a brain-computer interface fordetecting a neural activity-based input on the 3D model.

In an embodiment, the sensor includes a first input point for input froma first individual and a second input point for input from a secondindividual.

In an embodiment, the sensor includes an optical sensor for detecting agesture-based input, a facial-expression-based input, or an oculardilation-based input.

In an embodiment, the system includes a scanner in communication withthe computer readable medium for acquiring data of the scanned feature.

In an embodiment, the scanner includes an intra-oral scanner foracquiring data of the dental arch.

In an embodiment, the scanner includes an extraoral scanner foracquiring data of the portion of the face of the subject individual.

In an embodiment, the scanner includes an optical scanner.

In an embodiment, the scanner includes an ultrasonographic scanner.

In an embodiment, the system includes a muscle activity sensor formeasuring muscle activity of the individual's jaw.

In an embodiment, the muscle activity sensor includes anelectromyography module.

In an embodiment, the processor is in operative communication with thescanner for causing the scanner to acquire data for modelling thescanned feature; and the muscle activity sensor is in communication withthe processor for directing the scanner to acquire data for modellingthe scanned feature when the muscle activity is at a selected value.

In an embodiment, the selected value is indicative of a rest position.

In a further aspect, the present disclosure provides a method ofdesigning a dental appliance for a subject individual including:displaying a 3D model on a 3D display, the 3D model including: a scannedfeature including a dental arch of the subject individual and a portionof a face of the subject individual and the arch for relating the archto the face; and an augmented reality feature including a dentalappliance for the subject individual; detecting an input with a motionsensor; modifying the dental appliance in response to the gesture-basedinput to provide a modified dental appliance; repositioning the scannedfeature in response to the modified dental appliance; and updating the3D model in response to the modified dental appliance and therepositioned scanned feature to provide an updated 3D model.

In an embodiment, the input includes a gesture-based input.

In a further aspect, the present disclosure provides a method ofdesigning a dental appliance for a subject individual including:displaying a 3D model on a 3D display, the 3D model including: a scannedfeature including a dental arch of the subject individual and a portionof a face of the subject individual and the arch for relating the archto the face; and an augmented reality feature including a dentalappliance for the subject individual; detecting an input with an opticalsensor; modifying the dental appliance in response to the gesture-basedinput to provide a modified dental appliance; repositioning the scannedfeature in response to the modified dental appliance; and updating the3D model in response to the modified dental appliance and therepositioned scanned feature to provide an updated 3D model.

In an embodiment, the input includes a gesture-based input.

In an embodiment, detecting the input includes tracking eye movements.

In an embodiment, the input includes a facial expression.

In a further aspect, the present disclosure provides a method ofdesigning a dental appliance for a subject individual including:displaying a 3D model on a 3D display, the 3D model including: a scannedfeature including a dental arch of the subject individual and a portionof a face of the subject individual and the arch for relating the archto the face; and an augmented reality feature including a dentalappliance for the subject individual; detecting an input with abrain-computer interface; modifying the dental appliance in response tothe neural activity-based input to provide a modified dental appliance;repositioning the scanned feature in response to the modified dentalappliance; and updating the 3D model in response to the modified dentalappliance and the repositioned scanned feature to provide an updated 3Dmodel.

In a further aspect, the present disclosure provides a system fordesigning a dental appliance for a subject individual including: acomputer readable medium having a 3D model stored thereon, the 3D modelincluding a scanned feature including a dental arch of the subjectindividual and a portion of a face of the subject individual and thearch for relating the arch to the face, and an augmented reality featureincluding a dental appliance for the subject individual; a 3D displayfor displaying the 3D model; a motion sensor for detecting an input onthe 3D model; a processor operatively connected with the computerreadable medium for processing the 3D model, operatively connected withthe motion sensor for receiving the gesture-based input, and with the 3Ddisplay for displaying the 3D model, the processor configured andadapted to: modify the dental appliance in response to the gesture-basedinput to provide a modified dental appliance; reposition the scannedfeature in response to the modified dental appliance to provide arepositioned scanned feature; update the 3D model in response to themodified dental appliance and the repositioned scanned feature toprovide an updated 3D model; and display the updated 3D model on the 3Ddisplay.

In a further aspect, the present disclosure provides a system fordesigning a dental appliance for a subject individual including: acomputer readable medium having a 3D model stored thereon, the 3D modelincluding a scanned feature including a dental arch of the subjectindividual and a portion of a face of the subject individual and thearch for relating the arch to the face, and an augmented reality featureincluding a dental appliance for the subject individual; a 3D displayfor displaying the 3D model; a brain-computer interface for detecting aneural activity-based input on the 3D model; a processor operativelyconnected with the computer readable medium for processing the 3D model,operatively connected with the brain-computer interface for receivingthe neural activity-based input, and operatively connected with the 3Ddisplay for displaying the 3D model, the processor configured andadapted to: modify the dental appliance in response to the gesture-basedinput to provide a modified dental appliance; reposition the scannedfeature in response to the modified dental appliance to provide arepositioned scanned feature; update the 3D model in response to themodified dental appliance and the repositioned scanned feature toprovide an updated 3D model; and display the updated 3D model on the 3Ddisplay.

In a further aspect, the present disclosure provides a computer readablemedium having instructions encoded thereon for: rendering a 3D modelincluding a scanned feature and an augmented reality feature, thescanned feature including a dental arch of a subject individual and aportion of a face of the subject individual and the arch for relatingthe arch to the face, and the augmented reality feature including adental appliance for the subject individual; detecting an input from asensor; modifying the dental appliance in response to the input toprovide a modified dental appliance; repositioning the scanned featurein response to the modified dental appliance to provide a repositionedscanned feature; updating the 3D model in response to the modifieddental appliance and the repositioned scanned feature to provide anupdated 3D model; and displaying the updated 3D model on a 3D display.

In an embodiment, the input includes a voluntary input.

In an embodiment, the voluntary input includes a gesture-based input.

In an embodiment, the gesture-based input includes gripping a feature ofthe 3D model on the 3D display and manipulating the feature.

In an embodiment, gripping the feature includes gripping the featurewith a hand.

In an embodiment, the feature includes dentition of the dentalappliance.

In an embodiment, manipulating the feature includes changing angulationof the dentition.

In an embodiment, the gesture-based input originates from a firstindividual.

In an embodiment, the gesture-based input originates from a firstindividual and a second individual.

In an embodiment, the sensor includes a motion sensor.

In an embodiment, the voluntary input includes a neural activity input,and the sensor includes a brain-computer interface.

In an embodiment, the neural activity input includes a conceptualizationof the modified dental appliance.

In an embodiment, the neural activity input includes a conceptualizationof modifying the dental appliance.

In an embodiment, conceptualization of modifying the dental applianceincludes conceptualizing gripping a feature of the 3D model on thedisplay with a hand and manipulating the feature.

In an embodiment, the feature includes dentition of the dentalappliance.

In an embodiment, manipulating the feature includes changing angulationof the dentition.

In an embodiment, the voluntary input includes a gesture-based input,and the sensor includes a motion sensor.

In an embodiment, the neural activity input includes neural activityinput from a first individual.

In an embodiment, the neural activity input includes neural activityinput from a first individual and a second individual.

In an embodiment, the input includes constraining at least a portion ofthe scanned feature to a target position, and the modified dentalappliance includes a modified feature which facilitates the targetposition.

In an embodiment, the target position includes a selectedmaxillomandibular relationship.

In an embodiment, the selected maxillomandibular relationship is at arest position, and the dentition provides a freeway space of between 1and 4 mm at the rest position.

In an embodiment, the selected maxillomandibular relationship is at aselected occlusal position, and the dentition provides occlusion at theselected maxillomandibular relationship.

In an embodiment, the modified feature includes dentition of the dentalappliance.

In an embodiment, the instructions encoded thereon include detecting aninvoluntary input with the sensor; modifying the dental appliance inresponse to the involuntary input to provide the modified dentalappliance; repositioning the scanned feature in response to the modifieddental appliance to provide the repositioned scanned feature; updatingthe 3D model in response to the modified dental appliance and therepositioned scanned feature to provide the updated 3D model; anddisplaying the updated 3D model on the 3D display.

In an embodiment, the involuntary input includes involuntary input froma first individual.

In an embodiment, the involuntary input includes involuntary input froma first individual and a second individual.

In an embodiment, the involuntary input includes a neural activity inputand the sensor includes a brain-computer interface.

In an embodiment, the involuntary input includes a change in a facialexpression and the sensor includes an optical sensor.

In an embodiment, the instructions encoded thereon include detecting aninvoluntary input from a first individual with the sensor; correlatingthe involuntary input with a preference criterion and with the modifieddental appliance to determine a preference of the first individual;modifying the modified dental appliance to provide a suggested dentalappliance correlated to the preference of the first individual;repositioning the scanned feature in response to the suggested dentalappliance to provide a suggested scanned feature; updating the 3D modelin response to the suggested dental appliance and suggested scannedfeature to provide a suggested 3D model; and displaying the suggested 3Dmodel on the 3D display.

In an embodiment, the preference criterion includes an emotional stateof the first individual.

In an embodiment, the preference criterion includes a voluntary input ofan individual.

In an embodiment, the involuntary input includes involuntary input froma second individual, and the preference criterion includes an emotionalstate of the second individual.

In an embodiment, the involuntary input includes a neural activity inputand the sensor includes a brain-computer interface.

In an embodiment, the involuntary input includes a change a facialexpression and the sensor includes an optical sensor.

In an embodiment, the involuntary input is in response to the updated 3Dmodel.

In an embodiment, the 3D model includes a saved position, the savedposition having a selected scanned feature of the face.

In an embodiment, the instructions encoded thereon include:repositioning the scanned feature to the saved position; updating the 3Dmodel in response to the saved position and repositioned the scannedfeature to provide a saved position 3D model; and displaying the savedposition 3D model on the 3D display.

In an embodiment, the scanned feature includes external feature data ofthe face for additional detail on the face in the 3D model.

In an embodiment, the external feature data of the face includes datafor including substantially the entire face in the 3D model.

In an embodiment, the instructions encoded thereon further includingacquiring data of the scanned feature with a scanner.

In an embodiment, acquiring data of the scanned feature includesoptically scanning the scanned feature.

In an embodiment, acquiring data of the scanned feature includesultrasonographically scanning the scanned feature.

In an embodiment, acquiring data of the scanned feature includesacquiring additional data of the scanned feature in response to theinput and updating the 3D model to include the additional data.

In an embodiment, acquiring additional data and updating the 3D model toinclude the additional data are each performed continuously andsubstantially in real-time.

In an embodiment, adoption of a facial expression by the individualresults in updating the 3D model to include the additional data, andwherein the additional data includes external feature data of theindividual adopting the facial expression.

In an embodiment, the input includes a neural activity input, and thesensor includes a brain-computer interface.

In an embodiment, acquiring data of the scanned features includesconfirming that the subject individual is at a maxillomandibularrelationship corresponding to a rest position for the individual andacquiring data of the face when the maxillomandibular relationship is atthe rest position.

In an embodiment, confirming that the subject individual is at amaxillomandibular relationship corresponding to the rest positionincludes measuring jaw muscle activity of the individual to confirm amaxillomandibular relationship having a minimum energy usage.

In an embodiment, measuring the jaw muscle activity includes applyingelectromyography to the individual.

In an embodiment, confirming that the subject individual is at amaxillomandibular relationship corresponding to the rest positionincludes exhausting jaw muscles of the individual.

In an embodiment, exhausting jaw muscles of the individual includesapplying transcutaneous electrical nerve stimulation to the jaw muscles.

In an embodiment, data for rendering the 3D model includes data of theface when the maxillomandibular relationship is at the rest position.

Other aspects and features of the present disclosure will becomeapparent to those ordinarily skilled in the art upon review of thefollowing description of specific embodiments in conjunction with theaccompanying figures

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present disclosure will now be described, by way ofexample only, with reference to the attached figures, in which featuressharing reference numerals with a common final two digits of a referencenumeral correspond to corresponding features across multiple figures(e.g. the processor 12, 112, 212, 312, 412, 512, 612, 712, 812, 912,1012, 1112, 1212, 1312, etc.).

FIG. 1 is a schematic of a system for displaying and manipulating a 3Dmodel of an edentulous individual;

FIG. 2 is a flow chart of a method for displaying and manipulating the3D model of FIG. 1;

FIG. 3 is the individual of FIG. 1 viewing the 3D model of FIG. 1;

FIG. 4 is the 3D model of FIG. 1 after manipulation of themaxillomandibular relationship;

FIG. 5 is the 3D model of FIG. 1 after manipulation of a proposed dentalappliance;

FIG. 6 is the individual manipulating the 3D model of FIG. 1;

FIG. 7 is the individual manipulating the 3D model;

FIG. 8 is the individual manipulating the 3D model;

FIG. 9 is the individual zooming in on the 3D model;

FIG. 10 is the individual zooming out from the 3D model;

FIG. 11 is the individual rotating the 3D model;

FIG. 12 is the individual increasing the size of one tooth in the 3Dmodel;

FIG. 13 is the individual decreasing the size of one tooth in the 3Dmodel;

FIG. 14 is the individual increasing the size of one tooth in the 3Dmodel;

FIG. 15 is the individual decreasing the size of one tooth in the 3Dmodel;

FIG. 16 is a schematic of a system for displaying and manipulating a 3Dmodel of an edentulous individual;

FIG. 17 is a schematic of a system for displaying and manipulating a 3Dmodel of an edentulous individual;

FIG. 18 is a schematic of a system for displaying and manipulating a 3Dmodel of a partially dentate individual;

FIG. 19 is a schematic of a system for displaying and manipulating a 3Dmodel of an edentulous individual;

FIG. 20 is the individual of FIG. 16 viewing the 3D model of FIG. 16;

FIG. 21 is the individual manipulating the 3D model;

FIG. 22 is the individual manipulating the 3D model;

FIG. 23 is the individual zooming in on the 3D model;

FIG. 24 is the individual zooming out from the 3D model;

FIG. 25 is the individual rotating the 3D model;

FIG. 26 is the individual increasing the size of one tooth in the 3Dmodel;

FIG. 27 is the individual decreasing the size of one tooth in the 3Dmodel;

FIG. 28 is schematic of a system for displaying and manipulating a 3Dmodel of an edentulous individual;

FIG. 29 is a flow chart of a method for displaying and manipulating the3D model of FIG. 28;

FIG. 30 is a schematic of a system for displaying and manipulating a 3Dmodel of an edentulous individual;

FIG. 31 is a schematic of a system for displaying and manipulating two3D models of an edentulous individual;

FIG. 32 is a schematic of a system for displaying and manipulating a 3Dmodel of an edentulous individual;

FIG. 33 is a schematic of a system for acquiring data to prepare a 3Dmodel of an edentulous individual, and displaying and manipulating a 3Dmodel;

FIG. 34 is a flow chart of a method for acquiring data for, displaying,and manipulating the 3D model of FIG. 33;

FIG. 35 is a schematic of a system for acquiring data to prepare a 3Dmodel of an edentulous individual, displaying and manipulating a 3Dmodel, and updating the 3D model;

FIG. 36 is the system of FIG. 35 after updating external features data;

FIG. 37 is a flow chart of a method for displaying, manipulating, andupdating the 3D model of FIG. 35;

FIG. 38 is a schematic of a system for acquiring data to prepare a 3Dmodel of an edentulous individual, displaying and manipulating a 3Dmodel, and updating the 3D model;

FIG. 39 is a schematic of a system for displaying and manipulating a 3Dmodel of an edentulous individual; and

FIG. 40 is a schematic of a system for acquiring data to prepare a 3Dmodel of an edentulous individual, displaying and manipulating a 3Dmodel, and updating the 3D model.

DETAILED DESCRIPTION

Generally, the present disclosure provides a method and system forobserving the aesthetic effect of changes in dentition of a dentalappliance or restoration during design of the appliance or restoration.

Current practice in the dental field is for a professional to assess anindividual's dental condition, and to recommend treatments if required.In aesthetic dentistry, a dental professional would present treatmentswhich require an appliance to an individual. Design of the appliance isprimarily the responsibility of the dental professional and the dentallab, with minimal input from the individual. Expensive mock ups ortry-ins can be made from moldable materials by time-consumingprocedures. For this reason, if a presented try-in or mock up is notdesirable, it is rare to create more than a few mockups until one isdecided on. The individual may desire an alternative, but as they arenot skilled dental lab technicians, they may not be able to fullycommunicate their desires and a “doctor knows best” mentality commonlyleaves the individual with a compromise result, not fully achievingtheir initial desires. Empowering the individual to design their ownrestoration is not a practical alternative as the education necessary todesign a dental restoration is significant.

A person skilled in designing dental restorations on current modelingsoftware typically requires days of training to correctly use andunderstand design software before becoming proficient. It is impracticalto train a layperson individual who requires a dental appliance on suchdental design software. Therefore a system which allows an averageindividual the ability to interact with dental design softwareimmediately and intuitively observe aesthetic results from changes in aproposed appliance would be desirable.

An individual for whom a dental appliance is being designed is typicallyinterested in having input into their resulting appearance with theappliance. When preparing an appliance, preliminary models are oftenprepared by moulding and casting, which is time consuming, expensive,and imprecise. Predicting the effects of a particular change on theresulting smile of the individual and effectively communicating theprediction is challenging. As a result, it is challenging to providemeaningful input to the individual as to their resulting appearance.Given the impact on the individual's appearance of their dentition,satisfaction as to resulting appearance is vital to positive treatmentresults.

A method which allows the individual to observe and evaluate a proposedappliance prior to the costly fabrication of try-ins or final prostheseswould be advantageous over current methods of treatment visualisation.Many current systems rely on software which overlays 2D images on a 3Dmodel. Such software is often specialized, difficult for most laypeopleto use and understand, and is not directed to real-time use with alayperson individual. Manipulation of dentition in such software is doneby rotating, tilting, and otherwise changing the position and angulationof individual teeth or groups of teeth to change the features of theappliance without affecting the resulting bite. In addition, thedentition can be switched out with other pre-modeled dentition. Thepresent disclosure provides methods and systems which include real-timeaugmented reality (“AR”) integrated with 3D modeling, dental designsoftware, 3D display, and sensors. The sensors may include a motionsensor or motion capture device (e.g. an optical motion sensor, eyetracking sensors such as the SMI Eye Tracking Glasses 2 Wireless system,hand-based motion capture devices such as CyberGlove systems, etc.) forreceiving inputs based on gestures (e.g. tracking hand gestures,tracking eye movements, etc.), other optical sensors, a brain-computerinterface (“BCI”) for receiving inputs based on neural activity, sensorsfor measuring pulse, temperature, or perspiration, or combinations ofmultiple tests used as a polygraph, or any other appropriate sensor.Hand-based sensors may also provide tactile feedback to simulatehandling of a corporeal 3D model.

A 3D model of a portion of the individual's head is used in the methodsand systems described herein. The 3D model includes empirical data ofthe individual's arches and of the individual's head which relates thearches to the head (e.g. data from a 3D intra-oral optical scanner forthe arches and data from a 3D extraoral optical scanner for the arch andhead together). The 3D model is displayed in three dimensions (e.g. byvisualization through an Oculus Rift virtual reality headset, a GoogleGlass device, a holographic projection, etc.). In addition to theportions based on empirical data, the 3D model includes an augmentedreality dental appliance. Both the individual's head and the augmentedreality dental appliance can be manipulated by the individual throughuse of the individual's hands or other gestures, or through a BCI.Motion sensors detect the movement of the individual's hands or othergestures to receive an input, or the BCI receives input. The 3D model ismanipulated based on the input.

Features of the appliance (e.g. teeth, festoons and other features ofgums in a denture, etc.) shown on the 3D model can be grasped andmanipulated, or the individual can imagine doing so in the case of BCI.In addition, well-known hand gestures can be used or imagined forrotation, zooming, and other common functions. Combining eye trackingand a BCI may allow real time synchronizing of data from both systemswith a common time stamp to correlate patterns of attention withemotional states.

Changes to the augmented reality dental appliance can be made inreal-time on the 3D model by the individual, or another user of thesystem (e.g. a health care practitioner or a trusted layperson assistingthe individual). The individual can then view the model from any angleand at varying zoom angles to observe the aesthetic result of thespecific sizes, shapes, and orientation of the teeth in the appliance.Through manipulation and modification of the augmented reality dentalappliance and the 3D model more broadly, the individual may providemeaningful input into the aesthetic result of changes to the particulardesign of the appliance. These manipulations may be accomplished throughintuitive hand or other gestures. The gestures may be actual gestures(in the case or motion sensors or BCI), or imagined gestures (in thecase of BCI). Using a BCI, imagined hand movements can be deciphered andmental commands given to manipulate the 3D model. AR features andelements of the model can, for example, be pushed, pulled, rotated,enlarged, shrunk, etc. to the individual's tastes. Additionally,elements like colour and shade can likewise be changed using mentalcommands. The individual's involuntary responses to the 3D model (e.g.emotional responses, etc.) may be accounted for and the modelautomatically updated in response to the individual's involuntaryresponses. The involuntary responses may be measured by sensors whichmeasure optical changes, neural activity (a BCI), pulse, temperature,perspiration, combinations of multiple tests used as a polygraph, or anyother appropriate indicator of an emotional state.

Interacting with the proposed appliance design is as simple as reachingout and moving a tooth or other feature of the model with theindividual's hand, or where a BCI is used, imagining reaching and movingthe feature or imagining the results of the change. As a result, onlybasic instruction to the individual is required. The design softwareincludes preset functional limits on permissible changes to dentition toensure that the resulting appliance has an appropriate bite with equalinterdigitation, and allows an appropriate physiological rest position.The resulting appliance would be functional, but the aesthetics would bein the hands of the individual, subject to constraints imposed to ensurethat the resulting dental appliance provides appropriate clearancebetween upper and lower dentition at rest, provides an appropriatemaxillomandibular relationship at occlusion, and is otherwise optimizedfunctionally. The aesthetic results of changes to the proposed appliancecan be verified in real time, and the individual is able to find adesign they are satisfied with. Alternatively, the individual mayrealize that their ideal design lies outside of functional limits, andtherefore is not achievable. The individual could then manipulate the 3Dmodel to prepare a suitable compromise between aesthetics andfunctionality. The 3D nature of the model and the display, including theability to view from any angle and with varying levels of opacity,provides the individual with an understanding of how they will look inthree dimensions with a proposed appliance.

The 3D model can also be updated to reflect different facialexpressions. Smiles derived from old pictures of preexisting teeth canbe input into the system to restore an individual's natural smile.Celebrity smiles can also be input to influence design options. Theindividual can express their desired design result without fear ofjudgment or criticism from dental professionals. Similarly, a secondindividual may participate and propose change to the appliance. Thepreset smile or other constrains may be set as aesthetic goals, and adental appliance modeled to reach or approximate the goals whilemaintaining functional constraints related to a bite. Additional datamay be acquired while manipulating the 3D model to provide empiricaldata at a given position, such as empirical data at an occlusal positionwith current dentures, of a smile, or of the individual at aphysiological rest position (“rest position”).

A 3D model prepared from empirical data acquired when the individual'smaxillomandibular relationship is at the rest position provides anaccurate representation of the individual's maxillomandibularrelationship at the rest position (in contrast with acquiring data at adifferent position and extrapolating to the rest position). Theindividual's actual rest position determines that of the 3D model. Therest position of the 3D model thereby accounts for the interrelationshipof all the entities within the stomatognathic system, including joints,muscles, nerves, gums, implants (if any), and teeth (if any), whichaffect the rest position. A 3D model prepared without any data of anindividual at rest position is less likely to reliably distinguish arest position from a habitual or other position.

The rest position is a position of the mandible in space relative to themaxilla (vertical, anterior-posterior, and lateral relative to the headin an upright postural position) along an isotonic path of mandibularclosure. At the rest position, jaw musculature, including the extensorand depressor muscles that move the mandible, is postured at a positionwherein it exerts a minimum of electrical activity. Expenditure ofenergy by the jaw musculature required to maintain the rest position isminimal compared to other positions along a path of mandible hinging. Inthe rest position, the individual's condyles are in a neutral,unrestrained position.

The rest position of an individual can be determined with reference tothe individual. The rest position cannot be determined on a mechanicaldevice that simulates mandibular movements, such as a dentalarticulator. A mandibular position, or maxillomandibular relationship,can be influenced by factors including postural problems of the head,neck cervical region, and back region. Internal derangements of thetemporomandibular joint, emotional factors and systemic health factorsof the individual can also contribute to a compromised mandibularposition. It is generally beneficial to account for these factors beforeestablishing a rest position. In some cases, failure to account forthese factors results in an erroneous rest position. For example, afactor may have to be addressed or removed before establishing a restposition, which may be used to extrapolate to a bite registration. Inanother example, a factor may further complicate extrapolating restposition from other positions, increasing an advantage to acquisition ofdata of the individual at rest position.

A 3D model including empirical data at the rest position facilitatesaccurate determination of other potentially useful maxillomandibularrelationships. For example, the 3D model may be applied to jaw trackingand extraoral bite assessment of individuals lacking sufficientdentition to establish a bite, for example edentulous individuals. Thedata may facilitate determination of a natural position at which centricocclusion (“CO”; which occurs when an individual's teeth are at maximumintercuspation, and the individual's jaw is at a “CO position”) wouldoccur if the individual had sufficient dentition to establish a bite.The data may thus facilitate approximation of an optimal neuromuscularCO position. An estimated CO position may be applied to preparingdentures for individuals who do not have enough teeth to define a bite.

It is common for a denturist or other dental professional to establish aCO position when preparing an appliance. Where the individual lackssufficient dentition to establish the CO position, extrapolation isnecessarily required to determine an appropriate maxillomandibularrelationship in which CO should occur with an appliance. An edentulousindividual will lack sufficient dentition to establish the CO position.Some partially dentate individuals will also lack sufficient dentitionto establish CO, for example individuals with incisors but no molars.

Establishing a CO position based on the rest position when preparing anappliance may facilitate improvement and optimization of resultingdental function, stability, and harmony, of the stomatognathic systemincluding the appliance. Establishing the CO position based on the restposition may also facilitate one or more of the following:

-   -   optimization of the individual's occlusal scheme to a normal        occlusal scheme where a normal occlusal scheme will provide        appropriate functionality to the individual, or accounting for        any jaw relationship classification or malocclusion where the        individual's CO position may require as much;    -   optimization of dental aesthetics (including tooth shape,        contour, anatomy and morphology in both the anterior and        posterior regions);    -   optimization of facial cosmetics due to a more harmonious        muscular balance when an optimal physiologic mandibular position        is found; or    -   mitigation of possible musculoskeletal occlusal signs and        symptoms including:    -   headaches, ear congestion feelings, ringing in the ears,        pressure behind the eyes, teeth sensitivities, temporomandibular        joint noise, masticatory muscle tenderness, neck and shoulder        pain.

The rest position is a true rest position, in contrast with a habitualposition. The habitual position is an acquired maxillomandibularposition that may be anteriorly positioned along the condylartranslation pathway. In a given individual, the rest position and thehabitual position may coincide or be very close. However, the energyrequired by jaw musculature to maintain the habitual position is notnecessarily a minimum as is the rest position. The habitual position issometimes used as a starting point in determining a CO position inedentulous individuals. However, beginning with the habitual positionmay provide a less desirable outcome with respect to planning dentaltreatment than beginning with the rest position.

The 3D model is displayed by a 3D technique (e.g. Google Glass, OculusRift, Microsoft HoloLens, 3D television or monitor, holographicprojection, etc.). Gestures used to manipulate the 3D model may beintuitive and simple (e.g. gripping a tooth with the individual's handand rotating the tooth, etc.). As a result, the individual can easilymanipulate the dentition of a proposed appliance to observe theaesthetic impact of a given choice of dentition. Performing similarmanipulations on a two-dimensional display would require greaterproficiency and abstraction. Using the 3D display makes fine andspecific changes in the position of teeth accessible to the laypersonindividual, particularly where the individual is unaccustomed tothree-dimensional visualization and manipulation on a two-dimensionaldisplay (which would be more common in the elderly, who are a major userof dentures and other dental appliances). Similarly, language barriersbetween the individual and a dental professional are not a bar toreviewing and manipulating the 3D model. This may have application whendesigning dentures for individuals in impoverished or inaccessible areaswhere multiple visits with a professional is impractical.

The hands-free nature of the manipulation means that infections are lesslikely to spread through contact with a tablet surface, keyboards,mouse, or other physical interface device.

A second individual may manipulate the 3D model, and where a BCI orother sensor which receives inputs of data indicative of emotionalstates is used, the emotional responses of the individual, the secondindividual, or both may be applied to predict the preferences of theindividual, the second individual, or both. These preferences may beweighted to facilitate design of an appliance which both the individualand the second individual have a strong response to. The secondindividual may manipulate the 3D model in conjunction with theindividual, or the second individual may do so without the participationor input of the individual (e.g. where the individual is unable tocommunicate or effectively choose for themselves, etc.).

Limitations of some previous methods result from the analysis beingbased on two integrated 2D images of the subject and teeth. The twoimages do not share common features and cannot be used for generation ofa 3D model. The resulting compound image is not visible from any otherangles. Also lighting conditions during the taking of the digitalphotograph may be inconsistent, and accurate representation of toothshading in the representation to the subject's face, may not be fullyaccurate. Representation of the proposed prosthesis suffers from thelack of reference points between the existing denture or restoration andthe subject's face. In addition, without a true 3D model, functionallimits of design are more difficult to test and apply as constraints. Asa result, non-functional designs may be to be modeled without indicationof the potential problem. Finally, by placing a shield into a subject'smouth for a green screen, face and lip support which are changed,altering the resulting aesthetics which are to be modeled.

Previous green screen technology, such as the TruRx system, involvesoverlaying a projection of a proposed design on top of the individual'sdental arches. In contrast, the 3D model used in the methods and systemsdisclosed herein relates arches to facial structures. In the event thatthe arches and teeth are completely obscured by the lip in a givenposition, the 3D model remains capable of accurately representing theproposed appliance and its effect on aesthetics.

A benefit of using a 3D model relates to the resting lip line. At theresting lip line, the lip is relaxed and the majority, often theentirety, of the teeth are not visible. By application of extraoralstructures (e.g. facial features, etc.) in addition to intraoralfeatures (e.g. the dentition proposed by the augmented reality), the 3Dmodel provides an accurate depiction of the effects of teeth on theexternal features even when the arches and appliance are not visible.

System

FIG. 1 is a system 10 for displaying and manipulating a 3D model 20 of asubject individual 30. The system 10 includes a processor 12 and acomputer readable medium 14. The 3D model 20 is rendered, manipulated,updated, and displayed through execution of instructions by theprocessor 12. The 3D model 20 is based on data maintained on thecomputer readable medium 14. The processor 12 and the computer readablemedium 14 may be on the same or device or separate devices, may be atseparate network locations, or any other suitable arrangement. Thefunctions of the processor 12 and the computer readable medium 14 may bedivided among multiple individual processors and computer readablemedia.

The system 10 includes a 3D display 16 in operative communication withthe processor 12 for displaying the 3D model 20 such that the individual30 can place their hands on the 3D model 20 to manipulate the 3D model20 on the 3D display 16, for example through intuitive gestures. In FIG.1, the individual 10 is manipulating the 3D model 20 with a hand 39. Thesystem 10 may allow the individual to change the position or view of the3D model 20, change selected features of the 3D model 20, or otherwisemanipulate the 3D model 20 using gestures directed at the 3D model 20 asshown on the 3D display 16. The gestures may include gripping a portionof the 3D model 20 with a hand and applying similar hand gestures aswould be used if manipulating a physical model. Examples of suchmanipulations may also include changing the view of the 3D model 20shown on the display, such as rotating, panning, zooming in or out,changing the lighting conditions, etc.

The 3D display 16 is shown as an eyewear-style AR interface (e.g. GoogleGlass, Oculus Rift, Microsoft HoloLens, Meta Spaceglasses, etc.).Eyewear AR interfaces allow the 3D model 20 to display over the actualphysical environment from the perspective of the individual 30. The 3Ddisplay 16 projects a compound environment, allowing the individual 30to see the 3D model 20 in three dimensions and with real-time updates.The eyewear-style 3D display 16 is interchangeable with any displaydevice that provides a perception to the individual 30 that the 3D model20 is in front of their eyes and can be manipulated with their hands orother commands, and viewed from multiple angles.

The system 10 includes a motion sensor 18. The motion sensor 18 detectsgestures of the individual 30 (e.g. movements of the hands, head, feet,etc.). The gestures result in inputs of first input data 60, which isprovided to the processor 12. The first input data 60 includes voluntaryaction data 62 corresponding to gestures of the individual 30 detectedby the motion sensor 18. The motion sensor 18 monitors the motion,location, position, and angle of the gestures of the individual 30,allowing the individual 30 to manipulate of the 3D model 20 on the 3Ddisplay 16. The motion sensor 18 may detect motion based on any suitabledata (e.g. optical, Doppler radar, passive IR, tomographic, combinationsthereof, etc.).

Other sensors may be included in the system 10 in addition to the motionsensor 18 or in place of the motion sensor 18 to allow the individual 30to interact with the 3D model 20 and the dental design softwareotherwise than through gestures (e.g. by using eye movements, voicecommands, facial expressions, etc.) (not shown) to provide the voluntaryaction data 62 that can be interpreted by the processor 12. Such sensorsmay be based on capture of optical data or other forms of data from theindividual 30 (e.g. the optical sensor 1296 of FIG. 39, etc.).

The 3D model 20 may also be manipulated in response to voluntary actiondata from a person other than the individual 30 (e.g. the system 610 ofFIG. 30, etc.).

FIG. 2 shows a method 80 of working with a 3D model. The method 80includes displaying the 3D model 82, receiving an input 84, and updatingthe 3D model 86 in response to receiving an input 84. The method 80 maybe performed using the system 10. Displaying the 3D model 82 andupdating the 3D model 86 may be completed on the 3D display 16 byexecution of instructions by the processor 12 using data stored in thecomputer readable medium 14. Receiving an input 84 may include detectionof a hand gesture by the motion sensor 18 of the system 10, othervoluntary inputs, involuntary inputs, or combinations of inputs.

Components of 3D Model

FIG. 3 shows the individual 30 viewing the 3D model 20. The 3D model 20is modeled based on scanned features data 40 and augmented reality data50. The 3D model 20 includes subject features 22 and a proposed dentalappliance 24. The subject features 22 include modeled arches 21 andmodeled external features 23. Unlike the 3D display 16 of FIG. 1, whichapplies an eyewear-based in interface, the 3D model 20 of FIG. 3 isshown as a three-dimensional projector (e.g. holographic projector, theTelePresence system by Musion Das Hologram Ltd., the Digital Lightfieldsystem by Magic Leap, etc.) which functions in the absence of eyewear.Either of these types of displays, or any other suitable display, may beused as the 3D display 16.

The scanned features data 40 includes arches data 42, relational data44, and external features data 46. The scanned features data 40 isempirical data, which is acquired for example by scanning arches 32 ofthe individual 30 with an intraoral optical scanner (e.g. acquired usingthe system 910 of FIG. 33, etc.) and external features 34 of theindividual 30 with an extraoral optical scanner (e.g. acquired using thesystem 910 of FIG. 33, etc.). While additional scanned features data 40may provide additional accuracy to the 3D model 20 for a given position,the 3D model 20 can be manipulated to many positions based only oninitially-acquired scanned features data 40. Where the individual 30'sappearance does not change much between sessions with the 3D model 20,the same scanned features data 40 may be applied across multiplesessions. The scanned features data 40 may be acquired from an opticalscanner, ultrasound scanner, other suitable scanner, or other suitabledata acquisition technique applied to the external features 34.

The arches data 42 facilitates modeling of the maxillary and mandibulardental arches 32 of the individual 30, providing the modeled arches 21.

The external features data 46 facilitates modelling portions of theindividual 30's external features 34. The greater the amount of externalfeatures data 46, the more extensive are the modelled external features23, with correspondingly more expansive observation of aesthetic effectson the external features face than a 3D model which lacks the additionalexternal feature data (e.g. see the 3D model 120 in FIG. 16). Acquiringdata similar to arches data 42, relational data 44, and externalfeatures data 46, and preparing a model based on types of data isfurther discussed in WO 2013/071435, which shares an inventor with thisapplication. Features for acquiring data may be included, such as theextraoral scanners 992, 1092, 1192, or 1392 shown in FIGS. 33, 35, 38,and 40, respectively.

The relational data 44 includes data of the arches 32 and of theexternal features 34 (e.g. portions of the face, portions of the neck,etc.). The relational data 44 facilitates establishing a relationshipbetween the arches 32, and between the external features 34 and thearches 32. The relative positions of the arches 32 define amaxillomandibular relationship. The relational data 44 allows theexternal features 34 to be modeled based on the relative positions ofthe arches 32 in addition to being modeled based on the dentition 25.The maxillomandibular relationship at occlusion for a given proposeddental appliance 24 contributes to the appearance of the modeledexternal features 23. Given a constraint on a particularmaxillomandibular relationship at occlusion, the dentition 25 will drivethe appearance of the modelled external features 23 at occlusion withthe proposed dental appliance 24.

The relational data 44 also allows the maxillomandibular relationship ofthe 3D model 20 to be modeled based on the position of the externalfeatures 34. Constraints may be placed on how the modeled externalfeatures 23 are to look. The proposed dental appliance 24 will bemodeled to result in the selected appearance of the modeled externalfeatures 23. Constraints would also be applied to the proposed dentalappliance 24 to ensure that the maxillomandibular relationship atocclusion and at rest which the proposed dental appliance 24 results inare both appropriate for the individual 30. The modeled externalfeatures 23 selected for a resulting appearance may result from aposition included in the external features data 46, or be substantiallysimilar to a position included in the external features data 46.Empirical data of such a position may increase the effectiveness of the3D model 20 in providing the proposed dental appliance 24 with dentition25 and other features correctly selected for the individual 30. The restposition may be defined with empirical evidence, for example asdiscussed below and in WO 2013/071435, which shares an inventor withthis application, the external features data 46 may include empiricaldata of the individual 30 at the rest position. The system 1310 includesfeatures to facilitate acquiring empirical external features data 46 atthe rest position.

The relational data 44 facilitates manipulation of the maxillomandibularrelationship in the 3D model 20 while maintaining an accuraterelationship between the two modeled arches 21, and between the modeledarches 21 and the modeled external features 23. The relationships areaccurate in that the 3D model 20 conforms to relationships that arereflective of corresponding relationships in the individual 30 betweenthe arches 32, and between the arches 32 and the external features 34.

The augmented reality data 50 includes a representation of the proposeddental appliance 24. The proposed dental appliance 24 shown is a pair ofdentures. Other appliances may also be modelled (e.g. a single denture,a prosthetic, a restoration, etc.). The proposed dental appliance 24 ismodeled based on the augmented reality data 50 and overlaid on themodeled arches 21. The proposed dental appliance 24 results in amaxillomandibular relationship between the modeled arches 21 atinterdigitation facilitated by dentition 25 on the proposed dentalappliance 24. The maxillomandibular relationship, and the resultinglocations of the modeled arches 21 and the modeled external features 23,are informed by the scanned features data to represent in the 3D model20 the effects of the maxillomandibular relationship between the arches32 on the external features 34.

The proposed dental appliance 24 is based on a defined maxillomandibularrelationship appropriate for the individual 30 (e.g. providingappropriate occlusal and rest positions, etc.) and condular angles whichdefine movement direction from the bite position.

When modeling the proposed dental appliance 24 in real time with AR, averification procedure may facilitate the 3D model 20 accuratelymodelling the proposed maxillomandibular relationship position to alignwith the observed movement of the individual 30. With no dentalappliances worn, the individual moves their jaw (e.g. in a regularchewing function, etc.). The observed data can be compared to the 3Dmodel 20 and if inconsistencies are discovered, the 3D model 20 can becorrected, with the maxillomandibular occlusion position or condularangles as useful landmarks when comparing the movement of the 3D model20 to the observed movements of the individual 30. The verificationprocedure may be based on the external feature data 46. The verificationprocedure may also be based on additional external feature data 46acquired using, for example, the system 910 (FIG. 33), the system 1010(FIG. 35), the system 1110 (FIG. 37), the system 1310 (FIG. 40), etc.

Manipulation of 3D Model

The system 10 facilitates intuitive manipulation of the 3D model 20 bythe individual 30, who may be a lay person. Manipulation may includechanging perspective of the 3D model 20. Manipulation may includechanging the position of the subject features 22, altering the proposeddental appliance 24, changing the underlying external feature data 46,or changing the underlying augmented reality data 50, to facilitateobserving the resulting effects on the aesthetics of the 3D model 20,particularly with respect to the modeled external features 23.Constraints may be applied to the maxillomandibular position atocclusion, the spacing of the dentition 25 at the rest position, theappearance of the modeled external features 23, combinations of thesefeatures, or other appropriate features depending on the goals ofdesigning the proposed dental appliance 24.

When applying constrains to the spacing of the dentition 25 at the restposition, a freeway space of between 1 and 4 mm may be defined, forexample a freeway space of about 2 mm. The freeway space is theclearance between the dentition 25 on upper and lower portions of theproposed dental appliance at the rest position. Excessive orinsufficient freeway space each distort facial appearances. Excessivefreeway space (“over-closed”) causes the mandible and lips to protrudeand have a ‘collapsed’ or ‘frowning’ appearance. Insufficient freewayspace (“over-opened”) causes the face to elongate, this causes the lipsto appear thinned and stretched out and the face has a generaluncomfortable look. This is due to the strain of the facial muscleswhich cannot rest as they are engaged in an attempt to close to theproper dimension. If an individual presents in an over-opened orover-closed state, the present methods and systems could be used todetermine how to change the maxillomandibular relationship to achieve adesired external appearance and an appropriate rest position andocclusal position.

FIG. 4 shows an updated 3D model 20 a having a differentmaxillomandibular relationship than the 3D model 20. The updated 3Dmodel 20 a results from manipulation of the subject features 22. Thedifferent maxillomandibular relationship may result in repositioning ofthe modeled external features 23, providing repositioned modeledexternal features 23 a (and similarly, the subject features 22 arerepositioned to repositioned subject features 22 a). Themaxillomandibular relationship of the 3D model 20 may be manipulated asa result of gestures or other input directed at the 3D model 20 asdisplayed on the 3D display 16 and detected by the motion sensor 18.Through application of the relational data 44, the positions of themodeled arches 21 relative to each other and to the modeled externalfeatures 23 may be updated in the 3D model 20 following a change in themaxillomandibular relationship to the updated 3D model 20 a.

FIG. 5 shows an updated 3D model 20 b including a modified dentalappliance 24 b resulting from manipulation of the proposed dentalappliance 24 and modeled based on modified augmented reality data 50 b.The modified dental appliance 24 b may result in a differentmaxillomandibular relationship at occlusion than the proposed dentalappliance 24. The different maxillomandibular relationship may result inrepositioning of the modeled external features 23, providingrepositioned modeled external features 23 b (and similarly, the subjectfeatures 22 are repositioned to repositioned subject features 22 b). Inaddition, the modified dental appliance 24 b may have the samemaxillomandibular relationship at occlusion as the proposed dentalappliance 24, but may nonetheless result in differing positions andappearances of the modeled external features 23, providing therepositioned modeled external features 23 b. The size, orientation,shape, colour tone, and any other appropriate features of the proposeddental appliance 24 and its components may also be updated to providethe modified dental appliance 24 b. Structural changes to dentition 25or other features of the proposed dental appliance 24 may have an effecton the subject features 22 in that the maxillomandibular relationship atthe rest position, the occlusion position, or other selected referencepoints change following changes to the proposed dental appliance 24. Theother features may include the interface between the proposed dentalappliance 24 and the modeled arches 21 or other aspects of the proposeddental appliance 24 which determine how the proposed dental appliance 24will sit on the modeled arches 21. The modeled external features 23 mayalso be manipulated to result in a new maxillomandibular relationshipwhich provides or approximates the selected position of the modeledexternal features 23. Changes to the proposed dental appliance 24 may beconstrained within preset limits defined by the individual 30 or asecond individual (e.g. the second individual 690 in FIG. 30, etc.).Such constraints would typically be to provide a physiologicallyappropriate rest position or occlusion position.

FIG. 6 shows the individual 30 directly interacting with the 3D model20. The individual 30 is gesturing as if to grip the 3D model 20 shownon the 3D display with the hand 39. The maxillomandibular relationshipbetween the modeled arches 21, the modeled external features 23, and theproposed dental appliance 24 may each be manipulated, and the resultingeffects on the 3D model 20 calculated by the processor 12. The 3D model20 is updated to account for differences in the subject features 22, theproposed dental appliance 24, or both. Corresponding differences on theexternal features 34 of the individual 30 are reflected by changes inthe modeled external features 23 and the corresponding portions of the3D model 20. Gripping a portion of the 3D model 20 and manipulating the3D model 20 with intuitive gestures updates the 3D model 20 in realtime, facilitating comparing the effects of these changes on aestheticsof the modeled external features 23, the dentition 25, or other aspectsof the 3D model 20.

Saved positions 26 for the model 20 may be available for viewing. Thesaved positions 26 may include, for example, saved facial expressions(e.g. smiles, frowns, etc.). In the saved positions 26, themaxillomandibular relationship of the modeled arches 21, the modeledexternal features 23, or both, are updated to reflect the savedpositions 26. The features of the proposed dental appliance 24 and thecorresponding augmented reality data 50 can be updated from the savedposition 26, and any resulting differences on the external features 34of the individual 30 reflected in the 3D model 20. The saved positions26 may include custom smiles with preset sizes and position of dentition25, for example to reflect celebrity smiles or a smile that theindividual 30 previously had.

Saved dental appliances 27 may also be available for including in the 3Dmodel 20. The individual 30 may choose between the prearranged saveddental appliances 27 and make custom alterations to the saved dentalappliances 27. In addition, the tooth shade can changed with a shadeselector 29. A new saved dental appliance 27 may be selected before orafter selection of a saved position 26 to facilitate observing therelative aesthetics of different saved dental appliances 27 at differentsaved positions 26.

The method 80 of FIG. 2 is applied when the 3D model 20 is manipulated.In response to receiving the first input data 60 from the motion sensor18, the processor 12 evaluates whether the first input data 60 resultsin updates to the 3D model 20, the augmented reality data 50, or theexternal feature data 46. The arches data 42 and the relational data 44will remain constant.

In addition to gripping the 3D model 20 as shown in FIG. 6, other handgestures can be used to manipulate the model 20.

FIGS. 7 and 8 show the individual 30 manipulating the model 20 withoutcontacting the model 20 with the hand 39. In FIG. 8, only the proposeddental appliance 24 portion of the 3D model 20 is shown. In this way,the dentition 25 and other aspects of the proposed dental appliance 24may be updated free of obstruction by the subject features 22. Oncechanges to the dentition 25 or other features of the proposed dentalappliance 24 are complete, the subject features 22 may be reintroducedinto the 3D model 20 as displayed on the 3D display 16 to facilitateobservation of the effects of the changes to the proposed dentalappliance 24 on the modeled external features 23.

In additional to facilitating manipulation of the proposed dentalappliance 24 and the subject features 22, the system may facilitateintuitive viewing from multiple angles, zooming, and other changes inperspective.

FIGS. 9 and 10 respectively show the individual 30 manipulating themodel 20 by moving two hands 39 together to zoom in and moving the hands39 apart to zoom out.

FIG. 11 illustrates the individual 30 rotating the 3D model 20 byrotating the hand 39.

FIGS. 12 and 13 illustrate the individual 30 respectively enlarging andshrinking a single tooth 28 by gripping the single tooth 28 on the 3Dmodel 20 and moving the hand 39 as if to stretch or compress the singletooth 28. Enlarging the single tooth 28 results in an enlarged singletooth 28 c, and a corresponding modified dental appliance 24 c andupdated 3D model 20 c. Similarly, shrinking the single tooth 28 resultsin a reduced single tooth 28 d and a corresponding modified dentalappliance 24 d and updated 3D model 20 d.

FIGS. 14 and 15 illustrate the individual 30 respectively enlarging andshrinking a single tooth 28 by hand gestures similar to those of FIGS.12 and 13 but which do not include gripping the 3D model 20.

The individual 30 can change tooth shape, size, shade and position ofthe proposed dental appliance 24 and observe the resulting changes onthe modeled external features 23 and on the 3D model 20 as a whole inreal-time. The 3D model 20 may be viewed from any angle or position,facilitating observation of changes to the 3D model 20 from variousangles. The 3D model 20 may be viewed with a first facial expression,the facial expression updated to an updated facial expression, and theexternal features 23 of the 3D model 20 updated accordingly. The updatedfacial expression may for example be selected from the saved positions26, prepared by manipulating the individual features 22 of the 3D model20, or may be prepared based on additional external features data 46which is acquired (e.g. with the extraoral scanners 992, 1092, 1192, or1392 shown in FIGS. 33, 35, 38, and 40, respectively). The positions ofdentition 25 on the proposed dental appliance 24 are limited withinpreset parameters which are selected to maintain a selected bite, so asto not allow the dentition 24 to be arranged such that the bite isoutside of functional limits.

Data Included in the 3D Model

FIG. 16 shows a system 110 in which the scanned features data 40 lacksthe external feature data 46 and the 3D model 120 lacks the modeledexternal features 23. The scanned features data 140 includes only thearches data 142 and the relational data 144.

FIG. 17 shows a system 210 wherein the 3D model 220 includes denturedata 248 for modeling the individual's dentures 238. The denture data248 is reflective of the individual 230's current dentures 238 and maybe acquired by scanning the individual 230's current dentures 238, forexample with an extraoral optical scanner 992 as included in the system910 of FIG. 33, etc. Augmented reality data 252 is based in part on thedenture data 248. The denture data 248 may inform the augmented realitydata 252 as a starting point to redesign dentures for the individual 230by presenting the proposed dental appliance 224 for review in the 3Dmodel 220 and modification. If the individual has more than one pair ofcurrent dentures, more than one set of dentures data 248 may be acquiredand a corresponding number of sets of augmented reality data 252 wouldbe provided. As with the system 10, the proposed dental appliance 224may be modified through manipulation of the 3D model 20 without alteringthe underlying augmented reality data 252. The maxillomandibularrelationship at occlusion of the proposed dental appliance 224 may bethe same as that of the individual's current dentures 238 or may bemodified from that of the individual's current dentures 238 (e.g. toprovide an appropriate spacing between the dentition 225 at the restposition, etc.).

FIG. 18 shows a system 310 wherein the individual 330 has partialdentition 336. The 3D model 320 includes a representation of the partialdentition 336, which is represented in partially dentate arches data343. The augmented reality data 350 used to prepare the proposed dentalappliance 324 takes into account the presence of the partial dentition336 as shown in the partially dentate arches data 343.

Brain-Computer Interface

FIG. 19 is a system 410 for displaying and manipulating the 3D model420.

FIG. 20 shows the system 410 and the data 440, 450 used to prepare the3D model 420.

The system 410 includes the processor 412, the computer readable medium414, and the 3D display 416. The individual 430 interacts with the 3Dmodel 420 through use of a brain-computer interface (“BCI”) 419. The BCI419 monitors a property of the individual 430's brain indicative ofneural activity to receive an input of neural activity data 466. Currentexamples of BCI systems which may be used as the BCI 419 include theInsight and EPOC/EPOC+ systems manufactured by Emotive and the MindWavesystem manufactured by NeuroSky, which are both based onelectroencephalography (“EEG”) and monitor electrical activity of thebrain. The BCI 419 may include any suitable BCI which supports real-timeuse by the individual 430, and is not restricted to a BCI applying EEG.Functional magnetic resonance imaging, which monitors blood flow in thebrain, and magneto electroencephalography, which monitors magneticfields resulting from electrical activity of the brain, may also be usedin the BCI 419 to receive the input of neural activity data 466.

The BCI 419 facilitates responsive updating of the 3D model 420 withoutthe need for a motion sensor, audio sensor, or other sensors based onactions of the individual 30 downstream of mental conceptualization ofthe action or of a change to the 3D model 420. In addition, facialexpressions such as blinking, winking, and smiling result in neuralactivity which may be received as inputs by the BCI 419 and can be usedto update the modeled external features 423. Such updates to the modeledexternal features may be to a saved position 426, other modifications tothe positions of the modeled external features 423, or throughacquisition of additional external feature data 446 (e.g. as in thesystem 1010 of FIG. 35, the system 1110 of FIG. 37, etc.). An actionactually taken by the individual 420 would include conceptualization ofthe action, facilitating use of the system 410 where the individual 420is unfamiliar with use of a BCI 419.

The inputs of neural activity data 466 include the voluntary action data462 corresponding to mental commands from the individual 430, which areprovided to the processor 412. Once calibrated to the individual 430 tothe extent necessary for the particular BCI 419 and processor 412, theneural activity data 466 includes the voluntary action data 462corresponding to thoughts, conceptualized gestures (including gestureswhich are physically made) conceptualized changes to the 3D model 420,or other mental or emotional activity of the individual 430 related toactions which may be taken in respect of the 3D model 420.

The voluntary action data 462 may correspond to motion, location,position, and angle of gestures of the individual 430 which are mentallyconceptualized by the individual 430 (e.g. the inputs may correspond toa series of common and intuitive conceptualized hand gestures whichallows the individual 430 to rotate, pan, zoom in and change thelighting conditions on the 3D model 420, etc.). Examples of suchgestures which may be conceptualized by the individual 430 and theresulting manipulations to the 3D model 420 may include the handgestures used to manipulate the proposed dental appliance 24 or theperspective on the model 20 shown in FIGS. 6 to 15 when using the system10, although with the system 410, the gestures would merely beconceptualized or imagined by the individual 430.

A system may also be prepared combining the features of the system 10and the system 410, providing a system with both motion sensors and aBCI (not shown). Input from the motion sensors and from the BCI may beweighted differently. In addition, motion sensor input may be used tocalibrate the BCI to the individual using such a system.

FIGS. 21 to 27 show the individual 430 manipulating the model 420 byconceptualizing the results of changes, resulting in voluntary actiondata 462 being received by the BCI 419 in the absence of actual gesturesor conceptualized gestures.

FIG. 21 shows the individual 430 manipulating the model 420 byconceptualizing a change to one of the saved positions 426, orconceptualizing one of the saved positions 426.

FIG. 22 shows the individual 430 manipulating the 3D model 420 byconceptualizing changes in the colour and shade of the teeth in theproposed dental appliance 424. In FIG. 22, only the proposed dentalappliance 424 portion of the model 420 is shown. The same approachapplies to selection of one of the saved dental appliances 427. The 3Ddisplay 416 may show the saved positions 426, saved dental appliances427, and the shade selector 429 for the individual 430 to focus on andchange through the voluntary action data 462. Alternatively, thesefeatures may be omitted from display on the 3D display 416 and theindividual need only conceptualize which saved position 426, saveddental appliance 427, or change in shade that the individual 430 wouldlike to see displayed on the 3D model 420.

FIGS. 23 to 25 respectively show the individual 430 manipulating themodel 420 by conceptualizing zooming in, zooming out, and rotating the3D model 420.

FIGS. 26 and 27 illustrate the individual 430 respectively enlarging andshrinking a single tooth 428 by conceptualizing selection of the singletooth 428 and changes in size of the single tooth 428.

Involuntary Response Data FIG. 28 shows a system 510 wherein the BCI 519receives inputs of the neural activity data 566 corresponding toemotions, reactions, or other involuntary responses of the individual530, providing involuntary response data 564 of the individual 530. Withreference to the involuntary response data 564 and with calibration, theBCI 519 facilitates assessment of the emotional states and reactions ofthe individual 530, which in turn may facilitate predicting preferencesof the individual 530 with respect to the proposed dental appliance 524.The BCI 519 facilitates receiving inputs corresponding to facialexpressions of the individual 530, either as voluntary action data 562or involuntary response data 564. The individual 530 need not actuallysmile or otherwise change facial expressions to trigger the update tothe 3D model 520. As with the neural activity data 566 of conceptualizedhand gesture inputs detected by the BCI 519, neural activity data 566corresponding to facial expressions need only be conceptualized, whethervoluntary action data 562 of voluntary facial expressions or involuntaryresponse data 564 of involuntary facial expressions. Facial expressionsoriginate as nerve impulses in the brain, which travel through motorneurons to a neuromuscular junction. Upon adequate stimulation, themotor neuron releases a flood of neurotransmitters that bind topostsynaptic receptors and trigger a response in the muscle fiber whichresults in muscle movement. The BCI 519 facilitates responsive andintuitive changes to the 3D model 520 based on emotions or other factors(e.g. request to view celebrity smile, voluntary or involuntary adoptingor conceptualizing a given facial expression, etc.).

The external feature data 546 may include empirical optical image datawhich is correlated to neural activity data 566 from the BCI 519 by theprocessor 512. The 3D model 520 may be updated in real-time in responseto the neural activity data 566 (e.g. to show a smile, frown, close oneeye, etc.). For example, the 3D model 520 may be updated to assume asaved position 526 corresponding to a smile in response to theindividual 530 smiling and generating neural activity data 566 from thesmile (whether voluntary action data 562 or involuntary response data564). Since the external feature data 546 has already been acquiredbased on the external features 534, the facial expression or otherupdate based on data from the BCI 519 would not necessarily correspondto the particular smile the individual 520 is presently making. Rather,the update to the 3D model 520 would be based on previously acquiredscanned features data 540 corresponding to the relevant command (e.g.smile, frown, close one eye, etc.). The previously acquired scannedfeatures data 540 may be included with the saved positions 526. Thesystem 1010 of FIG. 35 and the system 1110 of FIG. 36 include extraoraloptical scanners for adding additional data to the external featuresdata during use of the systems.

The processor 512 may be programmed to assess and quantify involuntaryresponse data 564 corresponding to different hypothetical dental designelements of the proposed dental appliance 524 (tooth shape and size,arrangement, shade, imperfections, etc.). The augmented reality data 550can be organized in a hierarchical order of preferences specific to theindividual 530 based on the involuntary response data 564. The order ofpreferences may be based on preference criteria such as an emotionalstate of the individual 530 or a voluntary input from the individual 530(or another individual; e.g. the second individual 690, the secondindividual 790, the second individual 890, etc.). A preference criterionbased on an emotional state equates the involuntary response data 564 toan emotional state to determine whether the 3D model 520 as displayedelicits a defined emotional response from the individual 530. Theresponse may be binary or more nuanced as described below in relation tostatistical models which may be applied to the involuntary response data564. A preference criterion based on a voluntary input from theindividual 530 measures the involuntary response data 564 againstguidelines or constraints which are voluntarily selected by a user ofthe system 510 (e.g. the dentition 525 not exceed a given width orspacing on the front teeth, overbite necessary, underbite necessary,etc.). The guidelines may be applied by the individual 530 or by anotherperson (e.g. the second individual 690, the second individual 790, thesecond individual 890, etc.).

The involuntary response data 564 may be fitted to a statistical model(e.g. an ordinal utility function may be estimated or intervalpreference data may be applied to provide an estimate of the componentutility part-worth functions which can be statistically developed,etc.). The processor 512 can use the statistical model to recommend aproposed dental appliance 524 with a greater probability of approval bythe individual 530, either by choosing a saved dental appliance 527 orby modifying the proposed dental appliance 524. Involuntary responsedata 564 facilitates assessment of reactions of the individual 530 todifferent arrangements of the proposed dental appliance 524 andquantification of the preferences of the individual 530. The statisticalmodel may be a simple like/dislike model, or may include a variety oftypes of responses (e.g. nostalgia, happiness, confidence, excitement,indifference, disgust, etc.) of differing magnitudes and weightingfactors, as has been well-known (e.g. with applications to preparingeffective advertising, etc.).

As with the system 410, a system may also be prepared combining thefeatures of the system 10 and the system 510, providing a system withboth motion sensors and a BCI which responds to inputs of involuntaryresponse data (not shown). Input from the motion sensors and from theBCI may be weighted differently. In addition, motion sensor input may beused to calibrate the BCI to the individual using such a system.Similarly, a system may be prepared combining the features of the system510 and the system 1210, providing a system with two streams ofinvoluntary response data from both a BCI and an optical or other sensorwhich responds to inputs of involuntary response data (not shown). Sucha system would provide a cross-check for calibrating detection of theinvoluntary response data for the particular individual using thesystem.

FIG. 29 shows a method 180 of working with a 3D model using bothvoluntary action data inputs and involuntary response data inputs (e.g.with the 3D model 520, the 3D model 620, the 3D model 1220, etc.). Themethod 180 includes displaying the 3D model 182, receiving an input ofvoluntary action data 183, and updating the 3D model 186 in response toreceiving an input of voluntary action data 183. The method 180 alsoincludes receiving an input of involuntary response data 185 and rankingthe current 3D model as displayed based on the involuntary response data187. Once ranking the 3D model as displayed based on the involuntaryresponse data 187 is complete, the method 180 applies an algorithm forassessing whether changes to the 3D model are likely to elicit a morepositive involuntary response 188. If the algorithm indicates thatchanges to the 3D model are not likely to elicit a more positiveinvoluntary response than the data resulting from receiving an input ofinvoluntary response data 185, the method 180 returns to displaying the3D model 182 until either of receiving an input of voluntary action data183 or receiving an input of involuntary response data 185 occurs.

If the algorithm indicates that changes to the 3D model are likely toelicit a more positive involuntary response than the data resulting fromreceiving an input of involuntary response data 185, the method proceedsto updating the 3D model 186 in response to receiving an input ofinvoluntary response data 185. In this case, updating the 3D model 186applies the changes that are likely to elicit a more positiveinvoluntary response than the data resulting from receiving an input ofinvoluntary response data 185. Updating the 3D model 186 in response toreceiving an input of involuntary response data 185 may be subject tothe user approving updating of the 3D model 186, or may occurautomatically upon determining whether changes to the 3D model arelikely to elicit a more positive involuntary response 188. The method180 may be performed using the processor 512 with reference to datastored in the computer readable medium 514 of the system 510, with thecorresponding features of the system 610, the system 1210, or with anysystem including a sensor to detect involuntary response data.

The step of receiving an involuntary input 185 may be from a BCI, anoptical scanner, other sensors for detecting pupil dilation, pulse, andother factors to assess emotional state, a polygraph combining suchsensors, or any other suitable approach to measuring the emotional stateand preferences of the individual providing the involuntary input.

In an application of the method 180, the individual 30 may have a betterreaction to, for example, the right side of their smile than to the leftside. The individual 30's preference for any design elements present onthe right side, but missing from the left side would be ranked, andproposed alterations could be recommended to a proposed dental applianceto provide increase the chances of a positive emotional reaction. Theseresponses may be binary response or more detailed in terms of how muchthe individual 30 likes the 3D model 20 as displayed compared to otheroptions, and design software could suggest design elements toincorporate or leave out based on preferences, design elements thatcannot coexist in the same design, constraints, and other factors.

The method 180 depends on the quality of data (e.g. from a BCI, opticalemotion detector, polygraph, etc.). An applicable algorithm may be basedon an ordinal utility function estimation, interval preference data(with a statistically-developed estimate of the component utilitypart-worth functions), etc. Developments of BCIs will further increasethe accuracy of such algorithms to provide every more accuratepreferential information.

Use by Two Individuals

FIG. 30 shows a system 610 wherein a second non-subject individual 690is engaged with the BCI 619. The BCI 619 receives the neural activitydata 666 from the individual 630, second neural activity data 676 fromthe second individual 690, or both. The neural activity data 666 mayinclude the voluntary action data 662, the involuntary response data664, or both. The second neural activity data 676 may include secondvoluntary action data 672, second involuntary response data 674, orboth.

The second voluntary action data 672 and second involuntary responsedata 674 may be received by the BCI 619 and applied as mental commands,emotional states, reactions or other neural activity of the secondindividual 690. In contrast, the system 510 receives voluntary actiondata 562 and involuntary response data 564 from the individual 530 orfrom a non-subject individual (not shown) only. However, contemporaneouscontrol by, and the opinion of, the second individual 690 (e.g. aspouse, partner, family member, confidant, etc.) is often also valuable(e.g. when the individual 630 is visually impaired, is uncommunicative,etc.). The processor 612 may be configured in a variety of manners todifferently respond to the neural activity data 666, the second neuralactivity data 676, or both. In this way the method 180 may be practicedwherein receiving an input of voluntary action data 183 and receiving aninput of involuntary response data 185 are each applied to the firstinput data 660, the second input data 670, or both, and with anyappropriate weighting as between the first input data 660 and the secondinput data 670.

The processor 612 may be configured to respond to both voluntary actiondata 662 and second voluntary action data 672, and weigh bothinvoluntary response data 664 and second involuntary response data 674when preparing a proposed dental appliance 624. This configuration wouldfacilitate control by, and ranking of the proposed dental appliance 624in response to the reactions of, both the individual 630 and the secondindividual 690. The involuntary response data 664 and second involuntaryresponse data 674 may be weighted differently.

The processor 612 may be configured to respond to both voluntary actiondata 662 and second voluntary action data 672, but weigh onlyinvoluntary response data 664 or second involuntary response data 674when preparing a proposed dental appliance 624. This configuration wouldfacilitate control by both the individual 630 and the second individual690, but provide suggestions and measure the involuntary responses ofonly one of the individual 630 or the second individual 690.

The processor 612 may be configured to respond to only one of voluntaryaction data 662 or second voluntary action data 672, but weigh bothinvoluntary response data 664 and second involuntary response data 674.This configuration would facilitate control by only one of theindividual 630 or the second individual 690, but would account for theinvoluntary responses of both the individual 630 and the secondindividual 690 when preparing a proposed dental appliance 624. Theinvoluntary response data 664 and second involuntary response data 674may be weighted differently.

The processor 612 may be configured to respond to only the secondvoluntary action data 672, and weigh only the second involuntaryresponse data 674. This configuration would facilitate control by thesecond individual 690, and would result in a proposed dental appliance624 selected with reference to only second involuntary response data674.

The processor 612 may be configured to respond to only the voluntaryaction data 662, and weigh only the second involuntary response data674. This configuration would facilitate control by only of theindividual 630, and would result in a proposed dental appliance 624selected with reference to only the second involuntary response data674.

The processor 612 may be configured to respond to only the secondvoluntary action data 672, and weigh only the involuntary response data664. This configuration would facilitate control by the secondindividual 690 only, and would result in a proposed dental appliance 624selected with reference to only the involuntary response data 664.

FIG. 31 is a system 710 wherein the individual 730 and the secondindividual 790 each provide input data to the BCI 719. The individual730 views the 3D model 720 on the 3D display 716 and manipulates the 3Dmodel 720 through the BCI 719 by inputting the first input data 760. Asecond 3D model 791 of the individual 730 is displayed for the secondindividual 790 on a second 3D display 717. The second individual 790views the second 3D model 791 on the second 3D display 717 andmanipulates the second 3D model 791 through the BCI 719 by inputting thesecond input data 770. Application of the first input data 760 withrespect to the 3D model 720 and of the second input data 770 withrespect to the second 3D model 791 may each be as described elsewhere inthis application. The individual 730 may provide voluntary action data762, involuntary response data 764, or both, for manipulating the 3Dmodel 720. Similarly and independently of the individual manipulatingthe 3D model 720, the second individual 790 may provide voluntary actiondata 772, involuntary response data 774, or both, for manipulating thesecond 3D model 791. Alternatively, the involuntary response data 764may be applied to a method similar to that of method 180 in respect ofthe second 3D model 791, or the second involuntary response data 774 maybe applied in respect of the 3D model 720.

FIG. 32 is a system 810 wherein the individual 830 and the secondindividual 890 each interact with the 3D model 820 through a motionsensor 818. The processor 812 may include instructions to allow both theindividual 830 and the second individual 890 to freely interact with the3D model 820, to bias towards one of the individual 830 and the secondindividual 890, to allow the individual 830 and the second individual890 to take turns, or any suitable arrangement.

A system may also be prepared combining the features of the system 810and the system 610, providing a system with both motion sensors and aBCI for one or both of the individual and the second individual (notshown). Such a system may also be prepared wherein separate 3D modelsare displayed and manipulated by the individual and the secondindividual, similarly to the system 710.

Acquisition of Scanned Feature Data

FIG. 33 is a system 910 which includes scanners in communication withthe computer readable medium 914 for acquiring the scanned features data940. An intraoral optical scanner 993 is for acquiring the arches data942 from the maxillary and mandibular dental arches 932. An extraoraloptical scanner 992 is for acquiring the relational data 944 from themaxillary and mandibular dental arches 932 and the external features934. The extraoral optical scanner 992 is also for acquiring theexternal features data 946 from the external features 934. Including thescanners in the system 910 facilitates acquiring the scanned featuresdata 940 and using the 3D model 920 at the same location. The scannedfeatures data 940 is provided from the extraoral optical scanner 992 andthe intraoral optical scanner 993 to the computer readable memory 914 byany method using a wired connection, wireless connection, transfer ofremovable media, etc.

The external features data 946 may be acquired with or without a dentureor other appliance in the mouth of the individual 930. Acquiring theexternal features data 946 with a denture or other appliance in themouth of the individual 930 which approximates the proposed dentalappliance 924 may improve modeling of the external features 934, asaffected by a proposed dental appliance 924. The additional externalfeatures data 946 may improve modeling accuracy of proposedrestorations. Existing dentures or bite rims could be placed in themouth during external facial data capture, at different facialexpressions, to improve the relationship between the proposed dentalappliance 924 and the resultant modelled external features 923.Temporary material (e.g. dental wax, etc.) could be added to existingdentures to approximate an improved denture which is closer to theexpected proposed dental appliance 924.

FIG. 34 shows a method 280 for acquiring data for, displaying, andmanipulating a 3D model. The method 280 includes acquiring the scannedfeatures data 281, displaying the 3D model 282, receiving an input 284,and updating the 3D model 286 in response to receiving an input 284. Themethod 280 may be performed using the system 910. Displaying the 3Dmodel 282 and updating the 3D model 286 may be completed on the 3Ddisplay 916 by execution of instructions by the processor 912 using datafrom acquiring the scanned features data 281, and which is stored in thecomputer readable medium 914. Receiving an input 284 may includedetection of a hand gesture by the motion sensor 918 of the system 910,other voluntary inputs, involuntary inputs, or combinations of inputs.

FIG. 35 is a system 1010 which includes the extraoral optical scanner1092 for updating the external features data 1046. The extraoral opticalscanner 1092 may be a standalone unit as shown, or may be included as anoutward-facing imaging sensor on the 3D display 1016 (e.g. a Leap orKinect camera system and a projected holography/overlay element, etc.).The extraoral optical scanner 1092 may be used while the 3D model 1020is being manipulated to acquire additional external features data 1046for use in the 3D model 1020.

FIG. 36 is the system 1010 acquiring updated external features data 1046e from the individual 1030 in a pose having updated external features1034 e, which differ in appearance from the external features 1034 ofFIG. 35 in that the individual 30 is adopting a different facialexpression. The extraoral optical scanner 1092 is scanning the updatedexternal features 1034 e and providing the resulting updated externalfeatures data 1046 e to the computer readable medium 1014 for use by theprocessor 1012 to render the updated 3D model 1020 e having updatedsubject features 1022 e, which include updated modeled external features1023 e. The 3D model 1020 e is based on empirical data of the updatedmodeled external features 1023 e, which may facilitate more accuratemodelling of the updated modeled external features 1023 e compared withmoving the 3D model 1020 to a facial expression approximating that shownby the updated external features 1034 e but without empirical data.

During use of the system 1010, the individual 1030 may decide that agiven facial expression as shown on the 3D model 1020 would benefit fromempirical data to more accurately reflect the appearance of theindividual 1030 at the given facial expression. The individual maychange their facial expression to the updated external features 1034 eand activate the extraoral optical scanner 1092 to acquire the updatedexternal features data 1046 e, which is stored in the computer readablemedium 1014. The processor 1012 updates the 3D model 1020 to include theupdated external features data 1046 e, providing the updated 3D model1020 e. The updated 3D model 1020 e may be saved as a saved position1026. The individual 1030 may include dentures on their maxillary andmandibular arches 1032 prior to acquiring the updated external featuresdata 1046 e where the facial expression would benefit from dentition(not shown).

FIG. 37 shows a method 380 of working with and updating a 3D model. Themethod 380 includes displaying the 3D model 382, receiving an input 384,and updating the 3D model 386 in response to receiving an input 384. Inaddition, the method 380 includes receiving additional external featuresdata 394 and updating the 3D model in response to the additionalexternal features data 395. The method 380 may be performed using thesystem 1010. Displaying the 3D model 382, updating the 3D model 386, andupdating the 3D model in response to the additional external featuresdata 395 may be completed on the 3D display 1016 by execution ofinstructions by the processor 1012 using data stored in the computerreadable medium 1014, including the updated external features data 1046e. Receiving an input 384 may include detection of a hand gesture by themotion sensor 1018 of the system 1010, other voluntary inputs,involuntary inputs, or combinations of inputs.

The system 1010 may be used to continuously acquire the updated externalfeatures data 1046 e, resulting in real-time updating of the 3D model1020 to reflect the current facial expression of the individual 1030.This application of the system 1010 results in practicing the method 380in real-time, and effectively allows the individual 1030 to view andmanipulate a real-time augmented reality mirror showing the model 1020in the same facial expression that the individual 1030 is currentlyholding, adjusted for the presence of the proposed dental appliance1024. The real-time data acquisition and modelling could be oncontinuously, or transiently applied as selected by the individual 1030.

FIG. 38 is a system 1110 including the extraoral optical scanner 1192and the BCI 1119. The BCI 1119 may facilitate predictive acquisition ofadditional external features data 1146 and updating of the 3D model 1120to include additional external features data 1146 in real time, or atgiven emotional or other states which affect the external features 1134that are to be reflected in the modeled external features 1123,resulting in empirical data of a given facial expression, facilitatingmore accurately modelling of the facial expression by the 3D model 1120.

The BCI 1119 may also facilitate a comparison of the neural activitydata 1166 with the additional external feature data 1146. The comparisonmay facilitate accurate correlation of the neural activity data 1166with emotional states that may be recognized in the additional externalfeatures data 1146. In addition, real-time updating applications of thesystem 1110 may be facilitated compared with non-BCI equipped systems,such as the system 1010. The BCI 1119 may provide feedback on emotionalresponse during precise moments of contemplation.

FIG. 39 is a system 1210 wherein the involuntary response data 1264 isreceived without application of a BCI. The involuntary response data1264 may be acquired through an optical sensor 1296 which detects facialexpressions and other movements of the individual 1230 relevant to theemotional state of the individual 1230. The optical sensor 1296 may bedirected to detecting microexpressions, pupil dilation, and otherreliable indicators for the emotional state of the individual 1230,alone or in combination. In addition, other scanners which are notoptical may receive the involuntary response data 1264 (not shown; e.g.a pulse meter, temperature gauges, etc.), and the involuntary responsedata 1264 may be received by a combination of multiple types of data(not shown; e.g. a polygraph of temperature, pulse, and pupil dilation,etc.). Other than the involuntary response data 1264 being acquiredwithout the use of a BCI, the system 1210 functions similarly to thesystem 510, including with respect to the involuntary response data 1264triggering a saved position 1226.

A system may also be prepared combining the features of the system 510and the system 1210, providing a system with both an optical sensor (orother suitable non-BCI sensor; e.g. a polygraph of temperature, pulse,and pupil dilation, etc.) and a BCI (not shown). Input from the BCI andfrom the other sensor may be weighted differently. In addition, inputfrom the other sensor may be used to calibrate the BCI to the individualusing such a system.

The same hardware may perform the functions of the motion sensor (e.g.the motion sensor 18, the extraoral optical scanner 992, and the opticalsensor 1296). Generally, scanners for acquiring the scanned featuresdata may be more costly and subject to additional engineeringbottlenecks compared with scanners for acquiring the first input data.However, a single scanner (optical or otherwise) may be appliedavailable to acquire the scanned features data, the voluntary actiondata, and the involuntary response data, without departing from themethods and systems described herein.

FIG. 40 shows a system 1310 wherein a muscle activity sensor 1397 isengaged with the individual 1330 to measure activity of the individual1330's jaw musculature. The muscle activity sensor 1397 may for examplebe an electromyography module. The system 1310 includes the extraoralscanner 1392 for acquiring additional external features data 1346. Themuscle activity sensor 1397 detects when the muscle usage in jaw musclesof the individual 1330 is at a minimum, and sends a signal to theprocessor 1312 to direct the extraoral scanner 1392 to acquireadditional external features data 1346. As such, acquisition of theexternal features data 1346 may be acquired at the rest position. Inaddition, a transcutaneous electrical nerve stimulation module may beapplied to the individual's jaw musculature to exhaust the jawmusculature and force the maxillomandibular relationship to the restposition.

The 3D model 1320 may be used to check bite information. For anedentulous individual 1330, this would be possible with either nothingin their mouth, only an upper bite rim, or upper and lower bite rims ordentures (so long as the intraoral objects do not contact each otherprior to the desired bite location). As the individual 1330 closes theirjaw, the processor 1312 would facilitate determination of whether theindividual 1330 is biting in the proper occlusal position, which couldbe used as further confirmation of the scanned features data 1340. Aswith WO 2013/071435, which shares an inventor with this application,electromyography to assess facial muscle activity at various positions,or transcutaneous electrical nerve stimulation to force the restposition, each facilitate acquisition of data in the rest position. Thisinformation could be considered when defining appropriate constraintswithin which the individual 1330 can make adjustments while keeping thephysiological aspects of the bite consistent.

In some cases, imaging includes only maxillary data only. No biteinformation is required to model only upper front teeth. This does notchange the data acquisition inputs, aside from foregoing the mandibularportion of the arches data and the portion of the relational data thatrelating to the mandibular arch.

Examples Only

In the preceding description, for purposes of explanation, numerousdetails are set forth to provide a thorough understanding of theembodiments. However, it will be apparent to one skilled in the art thatthese specific details are not required. In some instances, specificdetails are not provided as to whether the embodiments described hereinare implemented as a software routine, hardware circuit, firmware, or acombination thereof.

Embodiments of the disclosure can be represented as a computer programproduct stored in a machine-readable medium (also referred to as acomputer-readable medium, a processor-readable medium, or a computerusable medium having a computer-readable program code embodied therein).The machine-readable medium can be any suitable tangible, non-transitorymedium, including magnetic, optical, or electrical storage mediumincluding a diskette, compact disk read only memory (CD-ROM), memorydevice (volatile or non-volatile), or similar storage mechanism. Themachine-readable medium can contain various sets of instructions, codesequences, configuration information, or other data, which, whenexecuted, cause a processor to perform steps in a method according to anembodiment of the disclosure. Those of ordinary skill in the art willappreciate that other instructions and operations necessary to implementthe described implementations can also be stored on the machine-readablemedium. The instructions stored on the machine-readable medium can beexecuted by a processor or other suitable processing device, and caninterface with circuitry to perform the described tasks.

The above-described embodiments are intended to be examples only.Alterations, modifications and variations can be effected to theparticular embodiments by those of skill in the art without departingfrom the scope, which is defined solely by the claims appended hereto.

1. A method of designing a dental appliance for a subject individualcomprising: displaying a 3D model of the subject individual on a 3Ddisplay, the 3D model comprising: a scanned feature comprising a dentalarch of the subject individual, and a portion of a face of the subjectindividual and the arch for relating the arch to the face; and anaugmented reality feature comprising a dental appliance for the subjectindividual; detecting a voluntary input with a sensor; modifying thedental appliance in response to the voluntary input to provide amodified dental appliance; repositioning the scanned feature in responseto the modified dental appliance to provide a repositioned scannedfeature; updating the 3D model in response to the modified dentalappliance and the repositioned scanned feature to provide an updated 3Dmodel; and displaying the updated 3D model on the 3D display.
 2. Themethod of claim 1 wherein the voluntary input comprises a gesture-basedinput.
 3. The method of claim 2 wherein the gesture-based inputcomprises gripping a feature of the 3D model on the 3D display andmanipulating the feature.
 4. The method of claim 3 wherein gripping thefeature comprises gripping the feature with a hand.
 5. The method ofclaim 3 wherein the feature comprises dentition of the dental appliance.6. The method of claim 5 wherein manipulating the feature compriseschanging angulation of the dentition.
 7. The method of claim 2 whereinthe gesture-based input originates from the subject individual.
 8. Themethod of claim 2 wherein the gesture-based input originates from anon-subject individual.
 9. The method of claim 2 wherein the sensorcomprises a motion sensor.
 10. The method of claim 1 wherein thevoluntary input comprises a neural activity input, and the sensorcomprises a brain-computer interface.
 11. The method of claim 10 whereinthe neural activity input comprises a conceptualization of the modifieddental appliance.
 12. The method of claim 10 wherein the neural activityinput comprises a conceptualization of modifying the dental appliance.13. The method of claim 12 wherein conceptualization of modifying thedental appliance comprises conceptualizing gripping a feature of the 3Dmodel on the display with a hand and manipulating the feature.
 14. Themethod of claim 10 wherein the feature comprises dentition of the dentalappliance.
 15. The method of claim 14 wherein manipulating the featurecomprises changing angulation of the dentition.
 16. The method of claim10 wherein the voluntary input comprises a gesture-based input, and thesensor comprises a motion sensor.
 17. The method of claim 10 wherein theneural activity input comprises neural activity input from the subjectindividual.
 18. The method of claim 10 wherein the neural activity inputcomprises neural activity input from a non-subject individual.
 19. Themethod of claim 1 wherein the voluntary input comprises constraining atleast a portion of the scanned feature to a target position, and themodified dental appliance comprises a modified feature which facilitatesthe target position.
 20. The method of claim 19 wherein the targetposition comprises a selected maxillomandibular relationship.
 21. Themethod of claim 20 wherein the selected maxillomandibular relationshipis a rest position, and the dentition provides a freeway space ofbetween 1 and 4 mm at the rest position.
 22. The method of claim 20wherein the selected maxillomandibular relationship is at a selectedocclusal position, and the dentition provides occlusion at the selectedmaxillomandibular relationship.
 23. The method of claim 20 wherein themodified feature comprises dentition of the dental appliance.
 24. Themethod of claim 1 further comprising: detecting an involuntary inputwith the sensor; modifying the dental appliance in response to theinvoluntary input to provide the modified dental appliance;repositioning the scanned feature in response to the modified dentalappliance to provide the repositioned scanned feature; updating the 3Dmodel in response to the modified dental appliance and the repositionedscanned feature to provide the updated 3D model; and displaying theupdated 3D model on the 3D display.
 25. The method of claim 24 whereinthe involuntary input comprises involuntary input from the subjectindividual.
 26. The method of claim 24 wherein the involuntary inputcomprises involuntary input from a non-subject individual.
 27. Themethod of claim 24 wherein the involuntary input comprises a neuralactivity input and the sensor comprises a brain-computer interface. 28.The method of claim 24 wherein the involuntary input comprises a changein a facial expression and the sensor comprises an optical sensor. 29.The method of claim 1 further comprising: detecting an involuntary inputwith the sensor; correlating the involuntary input with a preferencecriterion and with the modified dental appliance to determine apreference of an individual; modifying the modified dental appliance toprovide a suggested dental appliance correlated to the preference of theindividual; repositioning the scanned feature in response to thesuggested dental appliance to provide a suggested scanned feature;updating the 3D model in response to the suggested dental appliance andsuggested scanned feature to provide a suggested 3D model; anddisplaying the suggested 3D model on the 3D display.
 30. The method ofclaim 29 wherein the preference criterion comprises an emotional stateof the individual.
 31. The method of claim 29 wherein the preferencecriterion comprises a voluntary input of the individual.
 32. The methodof claim 29 wherein the involuntary input comprises involuntary inputfrom the subject individual.
 33. The method of claim 29 wherein theinvoluntary input comprises involuntary input from a non-subjectindividual.
 34. The method of claim 29 wherein the involuntary inputcomprises a neural activity input and the sensor comprises abrain-computer interface.
 35. The method of claim 29 wherein theinvoluntary input comprises a change a facial expression and the sensorcomprises an optical sensor.
 36. The method of claim 29 wherein theinvoluntary input is in response to the updated 3D model.
 37. The methodof claim 1 wherein the 3D model comprises a saved position, the savedposition having a selected scanned feature of the face.
 38. The methodof claim 37 further comprising: repositioning the scanned feature to thesaved position; updating the 3D model in response to the saved positionand repositioned the scanned feature to provide a saved position 3Dmodel; and displaying the saved position 3D model on the 3D display. 39.The method of claim 1 wherein the scanned feature comprises externalfeature data of the face for additional detail on the face in the 3Dmodel.
 40. The method of claim 39 wherein the external feature data ofthe subject individual's face comprises data for including substantiallythe entire face of the subject individual's face in the 3D model. 41.The method of claim 1 further comprising acquiring data of the scannedfeature.
 42. The method of claim 41 wherein acquiring data of thescanned feature comprises optically scanning the scanned feature. 43.The method of claim 41 wherein acquiring data of the scanned featurecomprises ultrasonographically scanning the scanned feature.
 44. Themethod of claim 41 wherein acquiring data of the scanned featurecomprises acquiring additional data of the scanned feature in responseto the voluntary input and updating the 3D model to include theadditional data.
 45. The method of claim 44 wherein acquiring additionaldata and updating the 3D model to include the additional data are eachperformed continuously and substantially in real-time.
 46. The method ofclaim 44 wherein adoption of a facial expression by the subjectindividual results in updating the 3D model to include the additionaldata, and wherein the additional data includes external feature data ofthe subject individual adopting the facial expression.
 47. The method ofclaim 46 wherein the voluntary input comprises a neural activity input,and the sensor comprises a brain-computer interface.
 48. The method ofclaim 41 wherein acquiring data of the scanned features comprisesconfirming that the subject individual is at a maxillomandibularrelationship corresponding to a rest position for the subject individualand acquiring data of the face when the maxillomandibular relationshipis at the rest position.
 49. The method of claim 48 wherein confirmingthat the subject individual is at a maxillomandibular relationshipcorresponding to the rest position comprises measuring jaw muscleactivity of the subject individual to confirm a maxillomandibularrelationship having a minimum energy usage.
 50. The method of claim 49wherein measuring the jaw muscle activity comprises applyingelectromyography to the subject individual.
 51. The method of claim 48wherein confirming that the subject individual is at a maxillomandibularrelationship corresponding to the rest position comprises exhausting jawmuscles of the subject individual.
 52. The method of claim 51 whereinexhausting jaw muscles of the subject individual comprises applyingtranscutaneous electrical nerve stimulation to the jaw muscles.
 53. Themethod of claim 1 wherein data for displaying the 3D model includes dataof the face when the maxillomandibular relationship is at the restposition.
 54. A system for designing a dental appliance for a subjectindividual comprising: a computer readable medium for storing a 3Dmodel, the 3D model comprising a scanned feature comprising a dentalarch of the subject individual and a portion of a face of the subjectindividual and the arch for relating the arch to the face, and anaugmented reality feature comprising a dental appliance for the subjectindividual; a 3D display for displaying the 3D model; a sensor fordetecting a voluntary input; a processor operatively connected with thecomputer readable medium for processing the 3D model, with the sensorfor detecting the voluntary input, and with the 3D display fordisplaying the 3D model, the processor configured and adapted to: modifythe dental appliance in response to the voluntary input to provide amodified dental appliance; reposition the scanned feature in response tothe modified dental appliance to provide a repositioned scanned feature;update the 3D model in response to the modified dental appliance and therepositioned scanned feature to provide an updated 3D model; and displaythe updated 3D model on the 3D display.
 55. The system of claim 54wherein the sensor comprises a motion sensor for detecting agesture-based input on the 3D model.
 56. The system of claim 54 whereinthe sensor comprises a brain-computer interface for detecting a neuralactivity-based input on the 3D model.
 57. The system of claim 54 whereinthe sensor comprises a first input point for input from a firstindividual and a second input point for input from a second individual.58. The system of claim 54 wherein the sensor comprises an opticalsensor for detecting a gesture-based input, a facial-expression-basedinput, or an ocular dilation-based input.
 59. The system of claim 54further comprising a scanner in communication with the computer readablemedium for acquiring data of the scanned feature.
 60. The system ofclaim 59 wherein the scanner comprises an intra-oral scanner foracquiring data of the dental arch.
 61. The system of claim 59 whereinthe scanner comprises an extraoral scanner for acquiring data of theportion of the face of the subject individual.
 62. The system of claim59 wherein the scanner comprises an optical scanner.
 63. The system ofclaim 59 wherein the scanner comprises an ultrasonographic scanner. 64.The system of claim 59 further comprising a muscle activity sensor formeasuring muscle activity of the subject individual's jaw.
 65. Thesystem of claim 64 wherein the muscle activity sensor comprises anelectromyography module.
 66. The system of claim 64 wherein: theprocessor is in operative communication with the scanner for causing thescanner to acquire data for modelling the scanned feature; and themuscle activity sensor is in communication with the processor fordirecting the scanner to acquire data for modelling the scanned featurewhen the muscle activity is at a selected value.
 67. The system of claim66 wherein the selected value is indicative of a rest position.
 68. Acomputer readable medium having instructions encoded thereon for:rendering a 3D model comprising a scanned feature and an augmentedreality feature, the scanned feature comprising a dental arch of asubject individual and a portion of a face of the subject individual andthe arch for relating the arch to the face, and the augmented realityfeature comprising a dental appliance for the subject individual;detecting a voluntary input from a sensor; modifying the dentalappliance in response to the voluntary input to provide a modifieddental appliance; repositioning the scanned feature in response to themodified dental appliance to provide a repositioned scanned feature;updating the 3D model in response to the modified dental appliance andthe repositioned scanned feature to provide an updated 3D model; anddisplaying the updated 3D model on a 3D display.
 69. The computerreadable medium of claim 68 wherein the voluntary input comprises agesture-based input.
 70. The computer readable medium of claim 69wherein the gesture-based input comprises gripping a feature of the 3Dmodel on the 3D display and manipulating the feature.
 71. The computerreadable medium of claim 70 wherein gripping the feature comprisesgripping the feature with a hand.
 72. The computer readable medium ofclaim 70 wherein the feature comprises dentition of the dentalappliance.
 73. The computer readable medium of claim 72 whereinmanipulating the feature comprises changing angulation of the dentition.74. The computer readable medium of claim 69 wherein the gesture-basedinput originates from a first individual.
 75. The computer readablemedium of claim 69 wherein the gesture-based input originates from afirst individual and a second individual.
 76. The computer readablemedium of claim 69 wherein the sensor comprises a motion sensor.
 77. Thecomputer readable medium of claim 68 wherein the voluntary inputcomprises a neural activity input, and the sensor comprises abrain-computer interface.
 78. The computer readable medium of claim 77wherein the neural activity input comprises a conceptualization of themodified dental appliance.
 79. The computer readable medium of claim 77wherein the neural activity input comprises a conceptualization ofmodifying the dental appliance.
 80. The computer readable medium ofclaim 79 wherein conceptualization of modifying the dental appliancecomprises conceptualizing gripping a feature of the 3D model on thedisplay with a hand and manipulating the feature.
 81. The computerreadable medium of claim 77 wherein the feature comprises dentition ofthe dental appliance.
 82. The computer readable medium of claim 81wherein manipulating the feature comprises changing angulation of thedentition.
 83. The computer readable medium of claim 77 wherein thevoluntary input comprises a gesture-based input, and the sensorcomprises a motion sensor.
 84. The computer readable medium of claim 77wherein the neural activity input comprises neural activity input from afirst individual.
 85. The computer readable medium of claim 77 whereinthe neural activity input comprises neural activity input from a firstindividual and a second individual.
 86. The computer readable medium ofclaim 68 wherein the voluntary input comprises constraining at least aportion of the scanned feature to a target position, and the modifieddental appliance comprises a modified feature which facilitates thetarget position.
 87. The computer readable medium of claim 86 whereinthe target position comprises a selected maxillomandibular relationship.88. The computer readable medium of claim 87 wherein the selectedmaxillomandibular relationship is at a rest position, and the dentitionprovides a freeway space of between 1 and 4 mm at the rest position. 89.The computer readable medium of claim 87 wherein the selectedmaxillomandibular relationship is at a selected occlusal position, andthe dentition provides occlusion at the selected maxillomandibularrelationship.
 90. The computer readable medium of claim 87 wherein themodified feature comprises dentition of the dental appliance.
 91. Thecomputer readable medium of claim 68, the instructions encoded thereonfurther comprising: detecting an involuntary input with the sensor;modifying the dental appliance in response to the involuntary input toprovide the modified dental appliance; repositioning the scanned featurein response to the modified dental appliance to provide the repositionedscanned feature; updating the 3D model in response to the modifieddental appliance and the repositioned scanned feature to provide theupdated 3D model; and displaying the updated 3D model on the 3D display.92. The computer readable medium of claim 91 wherein the involuntaryinput comprises involuntary input from a first individual.
 93. Thecomputer readable medium of claim 91 wherein the involuntary inputcomprises involuntary input from a first individual and a secondindividual.
 94. The computer readable medium of claim 91 wherein theinvoluntary input comprises a neural activity input and the sensorcomprises a brain-computer interface.
 95. The computer readable mediumof claim 68 wherein the involuntary input comprises a change in a facialexpression and the sensor comprises an optical sensor.
 96. The computerreadable medium of claim 68, the instructions encoded thereon furthercomprising: detecting an involuntary input from a first individual withthe sensor; correlating the involuntary input with a preferencecriterion and with the modified dental appliance to determine apreference of the first individual; modifying the modified dentalappliance to provide a suggested dental appliance correlated to thepreference of the first individual; repositioning the scanned feature inresponse to the suggested dental appliance to provide a suggestedscanned feature; updating the 3D model in response to the suggesteddental appliance and suggested scanned feature to provide a suggested 3Dmodel; and displaying the suggested 3D model on the 3D display.
 97. Thecomputer readable medium of claim 96 wherein the preference criterioncomprises an emotional state of the first individual.
 98. The computerreadable medium of claim 96 wherein the preference criterion comprises avoluntary input of an individual.
 99. The computer readable medium ofclaim 96 wherein the involuntary input comprises involuntary input froma second individual, and the preference criterion comprises an emotionalstate of the second individual.
 100. The computer readable medium ofclaim 96 wherein the involuntary input comprises a neural activity inputand the sensor comprises a brain-computer interface.
 101. The computerreadable medium of claim 96 wherein the involuntary input comprises achange a facial expression and the sensor comprises an optical sensor.102. The computer readable medium of claim 96 wherein the involuntaryinput is in response to the updated 3D model.
 103. The computer readablemedium of claim 68 wherein the 3D model comprises a saved position, thesaved position having a selected scanned feature of the face.
 104. Thecomputer readable medium of claim 103, the instructions encoded thereonfurther comprising: repositioning the scanned feature to the savedposition; updating the 3D model in response to the saved position andrepositioned the scanned feature to provide a saved position 3D model;and displaying the saved position 3D model on the 3D display.
 105. Thecomputer readable medium of claim 68 wherein the scanned featurecomprises external feature data of the face for additional detail on theface in the 3D model.
 106. The computer readable medium of claim 105wherein the external feature data of the face comprises data forincluding substantially the entire face in the 3D model.
 107. Thecomputer readable medium of claim 68, the instructions encoded thereonfurther comprising acquiring data of the scanned feature with a scanner.108. The computer readable medium of claim 107 wherein acquiring data ofthe scanned feature comprises optically scanning the scanned feature.109. The computer readable medium of claim 107 wherein acquiring data ofthe scanned feature comprises ultrasonographically scanning the scannedfeature.
 110. The computer readable medium of claim 107 whereinacquiring data of the scanned feature comprises acquiring additionaldata of the scanned feature in response to the voluntary input andupdating the 3D model to include the additional data.
 111. The computerreadable medium of claim 110 wherein acquiring additional data andupdating the 3D model to include the additional data are each performedcontinuously and substantially in real-time.
 112. The computer readablemedium of claim 110 wherein adoption of a facial expression by thesubject individual results in updating the 3D model to include theadditional data, and wherein the additional data includes externalfeature data of the subject individual adopting the facial expression.113. The computer readable medium of claim 112 wherein the voluntaryinput comprises a neural activity input, and the sensor comprises abrain-computer interface.
 114. The computer readable medium of claim 107wherein acquiring data of the scanned features comprises confirming thatthe subject individual is at a maxillomandibular relationshipcorresponding to a rest position for the subject individual andacquiring data of the face when the maxillomandibular relationship is atthe rest position.
 115. The computer readable medium of claim 114wherein confirming that the subject individual is at a maxillomandibularrelationship corresponding to the rest position comprises measuring jawmuscle activity of the subject individual to confirm a maxillomandibularrelationship having a minimum energy usage.
 116. The computer readablemedium of claim 115 wherein measuring the jaw muscle activity comprisesapplying electromyography to the subject individual.
 117. The computerreadable medium of claim 114 wherein confirming that the subjectindividual is at a maxillomandibular relationship corresponding to therest position comprises exhausting jaw muscles of the subjectindividual.
 118. The computer readable medium of claim 117 whereinexhausting jaw muscles of the subject individual comprises applyingtranscutaneous electrical nerve stimulation to the jaw muscles.
 119. Thecomputer readable medium of claim 68 wherein data for rendering the 3Dmodel includes data of the face when the maxillomandibular relationshipis at the rest position.
 120. The method of claim 1 wherein thevoluntary input originates from at least two individuals.